Update README.md
Browse files
README.md
CHANGED
@@ -31,15 +31,15 @@ In order to leverage instruction fine-tuning, your prompt should be surrounded b
|
|
31 |
|
32 |
E.g.
|
33 |
```
|
34 |
-
text = """<s>[INST]
|
35 |
-
|
36 |
```
|
37 |
|
38 |
This format is available as a [chat template](https://huggingface.co/docs/transformers/main/chat_templating) via the `apply_chat_template()` method:
|
39 |
|
40 |
## Example Code
|
41 |
|
42 |
-
Running this code requires
|
43 |
|
44 |
```python
|
45 |
from transformers import AutoModelForCausalLM, AutoTokenizer
|
@@ -50,7 +50,7 @@ model = AutoModelForCausalLM.from_pretrained("dicta-il/dictalm2.0-instruct-GPTQ"
|
|
50 |
tokenizer = AutoTokenizer.from_pretrained("dicta-il/dictalm2.0-instruct-GPTQ")
|
51 |
|
52 |
messages = [
|
53 |
-
{"role": "user", "content": "
|
54 |
{"role": "assistant", "content": "ืืื, ืื ื ืื ืืืื ืืื ืืืคืืช ืืืฅ ืืืืื ืกืืื ืืจื. ืื ืืืกืืฃ ืืืืืง ืืช ืืืืืช ืื ืืื ื ืฉื ืืขื ืืืฆืืฅ ืืื ืื ืฉืื ื ืืืฉื ืืืืื!"},
|
55 |
{"role": "user", "content": "ืืื ืืฉ ืื ืืชืืื ืื ืืืืื ื?"}
|
56 |
]
|
@@ -60,7 +60,7 @@ encoded = tokenizer.apply_chat_template(messages, return_tensors="pt").to(device
|
|
60 |
generated_ids = model.generate(encoded, max_new_tokens=50, do_sample=True)
|
61 |
decoded = tokenizer.batch_decode(generated_ids)
|
62 |
print(decoded[0])
|
63 |
-
# <s> [INST]
|
64 |
# ืืื, ืื ื ืื ืืืื ืืื ืืืคืืช ืืืฅ ืืืืื ืกืืื ืืจื. ืื ืืืกืืฃ ืืืืืง ืืช ืืืืืช ืื ืืื ื ืฉื ืืขื ืืืฆืืฅ ืืื ืื ืฉืื ื ืืืฉื ืืืืื!</s> [INST] ืืื ืืฉ ืื ืืชืืื ืื ืืืืื ื? [/INST]
|
65 |
# ืืื, ืื ื ืืชืืื ืงื ืืืื ืืืืื ื ืืืชื:
|
66 |
#
|
|
|
31 |
|
32 |
E.g.
|
33 |
```
|
34 |
+
text = """<s>[INST] ืืืื ืจืืื ืืืื ืขืืื? [/INST]
|
35 |
+
ืืื, ืื ื ืื ืืืื ืืื ืืืคืืช ืืืฅ ืืืืื ืกืืื ืืจื. ืื ืืืกืืฃ ืืืืืง ืืช ืืืืืช ืื ืืื ื ืฉื ืืขื ืืืฆืืฅ ืืื ืื ืฉืื ื ืืืฉื ืืืืื!</s>[INST] ืืื ืืฉ ืื ืืชืืื ืื ืืืืื ื? [/INST]"
|
36 |
```
|
37 |
|
38 |
This format is available as a [chat template](https://huggingface.co/docs/transformers/main/chat_templating) via the `apply_chat_template()` method:
|
39 |
|
40 |
## Example Code
|
41 |
|
42 |
+
Running this code requires less than 5GB of GPU VRAM.
|
43 |
|
44 |
```python
|
45 |
from transformers import AutoModelForCausalLM, AutoTokenizer
|
|
|
50 |
tokenizer = AutoTokenizer.from_pretrained("dicta-il/dictalm2.0-instruct-GPTQ")
|
51 |
|
52 |
messages = [
|
53 |
+
{"role": "user", "content": "ืืืื ืจืืื ืืืื ืขืืื?"},
|
54 |
{"role": "assistant", "content": "ืืื, ืื ื ืื ืืืื ืืื ืืืคืืช ืืืฅ ืืืืื ืกืืื ืืจื. ืื ืืืกืืฃ ืืืืืง ืืช ืืืืืช ืื ืืื ื ืฉื ืืขื ืืืฆืืฅ ืืื ืื ืฉืื ื ืืืฉื ืืืืื!"},
|
55 |
{"role": "user", "content": "ืืื ืืฉ ืื ืืชืืื ืื ืืืืื ื?"}
|
56 |
]
|
|
|
60 |
generated_ids = model.generate(encoded, max_new_tokens=50, do_sample=True)
|
61 |
decoded = tokenizer.batch_decode(generated_ids)
|
62 |
print(decoded[0])
|
63 |
+
# <s> [INST] ืืืื ืจืืื ืืืื ืขืืื? [/INST]
|
64 |
# ืืื, ืื ื ืื ืืืื ืืื ืืืคืืช ืืืฅ ืืืืื ืกืืื ืืจื. ืื ืืืกืืฃ ืืืืืง ืืช ืืืืืช ืื ืืื ื ืฉื ืืขื ืืืฆืืฅ ืืื ืื ืฉืื ื ืืืฉื ืืืืื!</s> [INST] ืืื ืืฉ ืื ืืชืืื ืื ืืืืื ื? [/INST]
|
65 |
# ืืื, ืื ื ืืชืืื ืงื ืืืื ืืืืื ื ืืืชื:
|
66 |
#
|