add code example

README.md

@@ -12,7 +12,7 @@ This model is a medium-sized MoE implementation based on [cognitivecomputations/
 
 A 2x7b configuration offers better performance than a standard 7b model even if loaded in 4 bit.
 
-If this 2x7b model is loaded in 4 bit the hellaswag score is .8260 which is higher than the base model achieves on its own in full precision.
+If this 2x7b model is loaded in 4 bit the hellaswag score is .8270 which is higher than the base model achieves on its own in full precision.
 
 ## Prompt Format
 
@@ -37,12 +37,44 @@ Please give ideas and a detailed plan about how to assemble and train an army of
 <|im_start|>assistant
 ```
 
+## Models Merged
+
++ teknium/OpenHermes-2.5-Mistral-7B 
++ cognitivecomputations/dolphin-2.6-mistral-7b-dpo-laser
+  
 ## Code Example
+Switch the commented model definition to use in 4-bit. Should work with 9GB and still exceed the single 7B model by 5-6 points roughly
+
+```python
+from transformers import AutoTokenizer, AutoModelForCausalLM
+
+# Load tokenizer and model
+tokenizer = AutoTokenizer.from_pretrained("macadeliccc/laser-dolphin-mixtral-2x7b-dpo")
+model = AutoModelForCausalLM.from_pretrained("macadeliccc/laser-dolphin-mixtral-2x7b-dpo")
+# model = AutoModelForCausalLM.from_pretrained("macadeliccc/laser-dolphin-mixtral-2x7b-dpo", load_in_4bit=True)
+# Define the chat messages
+messages = [
+    {"role": "system", "content": "You are Dolphin, an AI assistant"},
+    {"role": "user", "content": "Hello, who are you?"}
+]
+
+# Apply chat template to input messages
+gen_input = tokenizer.apply_chat_template(messages, return_tensors="pt")
 
-TODO
+# Generate a response
+output = model.generate(**gen_input)
+
+# Decode the generated tokens to a string
+response = tokenizer.decode(output[0], skip_special_tokens=True)
+
+# Print the response
+print("Response:", response)
+```
 
 ## Eval
 
+**Full Precision**
+
 |  Tasks   |Version|Filter|n-shot| Metric |Value |   |Stderr|
 |----------|-------|------|-----:|--------|-----:|---|-----:|
 |arc_easy  |Yaml   |none  |     0|acc     |0.8413|±  |0.0075|
@@ -56,6 +88,20 @@ TODO
 |          |       |none  |     0|acc_norm|0.8303|±  |0.0088|
 |winogrande|Yaml   |none  |     0|acc     |0.7577|±  |0.0120|
 
+**4-bit (bnb)**
+
+|  Tasks   |Version|Filter|n-shot| Metric |Value |   |Stderr|
+|----------|-------|------|-----:|--------|-----:|---|-----:|
+|boolq     |Yaml   |none  |     0|acc     |0.8700|±  |0.0059|
+|hellaswag |Yaml   |none  |     0|acc     |0.6356|±  |0.0048|
+|          |       |none  |     0|acc_norm|0.8270|±  |0.0038|
+|openbookqa|Yaml   |none  |     0|acc     |0.3320|±  |0.0211|
+|          |       |none  |     0|acc_norm|0.4620|±  |0.0223|
+|piqa      |Yaml   |none  |     0|acc     |0.8123|±  |0.0091|
+|          |       |none  |     0|acc_norm|0.8259|±  |0.0088|
+|winogrande|Yaml   |none  |     0|acc     |0.7490|±  |0.0122|
+
+
 link to evaluation [colab](https://colab.research.google.com/drive/1FpwgsGzCR4tORTxAwUxpN3PcP22En2xk?usp=sharing)
 
 ## Citations