saheedniyi
commited on
Commit
•
81014a2
1
Parent(s):
8282088
Update README.md
Browse files
README.md
CHANGED
@@ -47,6 +47,15 @@ input_text = "What are the top places for tourism in Nigeria?"
|
|
47 |
formatted_prompt=input_text=f"### BEGIN CONVERSATION ###\n\n## User: ##\n{input_text}\n\n## Assistant: ##\n"
|
48 |
inputs = tokenizer(input_text, return_tensors="pt")
|
49 |
outputs = model.generate(**inputs.to("cuda"), max_new_tokens=512,pad_token_id=tokenizer.pad_token_id,do_sample=True,temperature=0.6,top_p=0.9,)
|
50 |
-
|
51 |
-
print(
|
52 |
-
```
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
47 |
formatted_prompt=input_text=f"### BEGIN CONVERSATION ###\n\n## User: ##\n{input_text}\n\n## Assistant: ##\n"
|
48 |
inputs = tokenizer(input_text, return_tensors="pt")
|
49 |
outputs = model.generate(**inputs.to("cuda"), max_new_tokens=512,pad_token_id=tokenizer.pad_token_id,do_sample=True,temperature=0.6,top_p=0.9,)
|
50 |
+
response=tokenizer.decode(outputs[0], skip_special_tokens=True)
|
51 |
+
print(response)
|
52 |
+
```
|
53 |
+
when using the model it is important to use the chat template that the model was trained on.
|
54 |
+
|
55 |
+
```
|
56 |
+
prompt = "INPUT YOUR PROMPT HERE"
|
57 |
+
formatted_prompt=input_text=f"### BEGIN CONVERSATION ###\n\n## User: ##\n{prompt}\n\n## Assistant: ##\n"
|
58 |
+
```
|
59 |
+
|
60 |
+
The model has a little tokenization issue and it's necessary to wtrite a function to clean the output to make it cleaner and more presentable.
|
61 |
+
**This issue shold be resolved in the next version of the model.**
|