File size: 3,098 Bytes
66514b4 0019f04 2443029 1e9c6cb 66514b4 2443029 66514b4 1e9c6cb 66514b4 1e9c6cb 66514b4 2443029 66514b4 1e9c6cb 2443029 1e9c6cb 66514b4 fc78757 66514b4 1e9c6cb 66514b4 1e9c6cb 66514b4 1e9c6cb 2443029 66514b4 1e9c6cb fc78757 1e9c6cb 2443029 66514b4 6d1e065 541c49f 2443029 6d1e065 cad4e7b |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 |
---
library_name: transformers
license: apache-2.0
pipeline_tag: text-generation
datasets:
- maywell/ko_Ultrafeedback_binarized
base model:
- meta-llama/Meta-Llama-3-8B-Instruct
---
# T3Q-Llama3-8B-Inst-sft1.0
## This model is a version of meta-llama/Meta-Llama-3-8B-Instruct that has been fine-tuned with SFT.
## Model Developers Chihoon Lee(chihoonlee10), T3Q
#### Transformers pipeline
```python
import transformers
import torch
model_id = "chlee10/T3Q-Llama3-8B-Inst-sft1.0"
pipeline = transformers.pipeline(
"text-generation",
model=model_id,
model_kwargs={"torch_dtype": torch.bfloat16},
device_map="auto",
)
messages = [
{"role": "system", "content": "You are a pirate chatbot who always responds in pirate speak!"},
{"role": "user", "content": "Who are you?"},
]
prompt = pipeline.tokenizer.apply_chat_template(
messages,
tokenize=False,
add_generation_prompt=True
)
terminators = [
pipeline.tokenizer.eos_token_id,
pipeline.tokenizer.convert_tokens_to_ids("<|eot_id|>")
]
outputs = pipeline(
prompt,
max_new_tokens=256,
eos_token_id=terminators,
do_sample=True,
temperature=0.6,
top_p=0.9,
)
print(outputs[0]["generated_text"][len(prompt):])
```
#### Transformers AutoModelForCausalLM
```python
from transformers import AutoTokenizer, AutoModelForCausalLM
import torch
model_id = "chlee10/T3Q-Llama3-8B-Inst-sft1.0"
tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(
model_id,
torch_dtype=torch.bfloat16,
device_map="auto",
)
messages = [
{"role": "system", "content": "You are a pirate chatbot who always responds in pirate speak!"},
{"role": "user", "content": "Who are you?"},
]
input_ids = tokenizer.apply_chat_template(
messages,
add_generation_prompt=True,
return_tensors="pt"
).to(model.device)
terminators = [
tokenizer.eos_token_id,
tokenizer.convert_tokens_to_ids("<|eot_id|>")
]
outputs = model.generate(
input_ids,
max_new_tokens=256,
eos_token_id=terminators,
do_sample=True,
temperature=0.6,
top_p=0.9,
)
response = outputs[0][input_ids.shape[-1]:]
print(tokenizer.decode(response, skip_special_tokens=True))
```
hf (pretrained=chlee10/T3Q-Llama3-8B-Inst-sft1.0), limit: None, provide_description: False, num_fewshot: 0, batch_size: None
```python
| Task |Version| Metric |Value | |Stderr|
|----------------|------:|--------|-----:|---|-----:|
|kobest_boolq | 0|acc |0.5114|± |0.0133|
| | |macro_f1|0.3546|± |0.0080|
|kobest_copa | 0|acc |0.6000|± |0.0155|
| | |macro_f1|0.5997|± |0.0155|
|kobest_hellaswag| 0|acc |0.4120|± |0.0220|
| | |acc_norm|0.5380|± |0.0223|
| | |macro_f1|0.4084|± |0.0219|
|kobest_sentineg | 0|acc |0.5063|± |0.0251|
| | |macro_f1|0.3616|± |0.0169|
``` |