Update README.md

README.md

@@ -7,7 +7,49 @@ language:
 
 
 This is a direct extraction of the 8 experts from [Mixtral-8x7b-Instruct-v0.1](https://huggingface.co/mistralai/Mixtral-8x7B-Instruct-v0.1), and placing them into the Deepseek-MoE Architecture.
-It is 2 experts per token. Performance is good, and is likely more malleable to training. This is our first experiment with expert extraction and modification, more to come. Enjoy.
+It is 2 experts per token. Performance is identical to instruct, if not a little better. Evals will come, It is more malleable to training. This is our first experiment with expert extraction and modification, more to come. Enjoy.
+
+
+```markdown
+## Instruction Format
+To leverage instruction fine-tuning, your prompts should be enclosed with `[INST]` and `[/INST]` tokens. The very first instruction should begin with a begin-of-sentence id, while subsequent instructions should not. Assistant generation will conclude with an end-of-sentence token id.
+
+### Example
+```plaintext
+text = "<s>[INST] What is your favourite condiment? [/INST]"
+"Well, I'm quite partial to a good squeeze of fresh lemon juice. It adds just the right amount of zesty flavour to whatever I'm cooking up in the kitchen!</s>"
+"[INST] Do you have mayonnaise recipes? [/INST]"
+```
+
+### Applying the Chat Template
+This format can be implemented using the `apply_chat_template()` method from the `transformers` library:
+
+```python
+from transformers import AutoModelForCausalLM, AutoTokenizer
+
+device = "cuda"  # the device to load the model onto
+
+# Load the model and tokenizer
+model = AutoModelForCausalLM.from_pretrained("mistralai/Mistral-7B-Instruct-v0.2")
+tokenizer = AutoTokenizer.from_pretrained("mistralai/Mistral-7B-Instruct-v0.2")
+
+# Define the conversation messages
+messages = [
+    {"role": "user", "content": "What is your favourite condiment?"},
+    {"role": "assistant", "content": "Well, I'm quite partial to a good squeeze of fresh lemon juice. It adds just the right amount of zesty flavour to whatever I'm cooking up in the kitchen!"},
+    {"role": "user", "content": "Do you have mayonnaise recipes?"}
+]
+
+# Apply chat template
+encodeds = tokenizer.apply_chat_template(messages, return_tensors="pt")
+model_inputs = encodeds.to(device)
+model.to(device)
+
+# Generate response
+generated_ids = model.generate(model_inputs, max_new_tokens=1000, do_sample=True)
+decoded = tokenizer.batch_decode(generated_ids)
+print(decoded[0])
+```
 
 Special Thanks: Eric Hartford, and Fernando Neto.