Update README.md

README.md

@@ -27,8 +27,73 @@ datasets:
 
 A fine-tuned version of [v2ray/Mixtral-8x22B-v0.1](https://huggingface.co/v2ray/Mixtral-8x22B-v0.1) model on the `philschmid/guanaco-sharegpt-style` dataset. This model has a total of 141b parameters with 35b only active. 
 
+
+## How to use it
+
+
+**Use a pipeline as a high-level helper:**
+```python
+from transformers import pipeline
+
+pipe = pipeline("text-generation", model="MaziyarPanahi/Goku-8x22B-v0.1")
+```
+
+**Load model directly:**
+```python
+
+from transformers import AutoTokenizer, AutoModelForCausalLM
+
+tokenizer = AutoTokenizer.from_pretrained("MaziyarPanahi/Goku-8x22B-v0.1")
+model = AutoModelForCausalLM.from_pretrained("MaziyarPanahi/Goku-8x22B-v0.1")
+```
+
+**Load via Adapter:**
+
+You can also use PEFT to just load the adapter if you already have one of these models downloaded: [v2ray/Mixtral-8x22B-v0.1](https://huggingface.co/v2ray/Mixtral-8x22B-v0.1) or [mistral-community/Mixtral-8x22B-v0.1](https://huggingface.co/mistral-community/Mixtral-8x22B-v0.1) (they are the same)
+
+```python
+# assuming you have already downloaded the 
+# resizing the vocab
+import torch
+from transformers import AutoModelForCausalLM, AutoTokenizer
+
+model_id="v2ray/Mixtral-8x22B-v0.1"
+peft_model_id = "~/.cache/huggingface/hub/models--MaziyarPanahi--Goku-8x22B-v0.1/adapter"
+
+tokenizer = AutoTokenizer. from_pretrained (peft_model_id)
+model = AutoModelForCausalLM. from_pretrained (model_id)
+# I have added 2 new tokens for ChatML template
+# this step is required if you are using PEFT/Adapter
+model.resize_token_embeddings (len (tokenizer))
+model.load_adapter(peft_model_id)
+
+# you can even have TextStreamer and a text-generation pipeline with your adapter
+streamer = TextStreamer(tokenizer)
+
+pipe = pipeline(
+    "text-generation",
+    model=model,
+    tokenizer=tokenizer,
+    max_new_tokens=750,
+    temperature=0.6,
+    do_sample=True,
+    top_k=50,
+    top_p=0.95,
+    repetition_penalty=1.1,
+    return_full_text=False,
+    add_special_tokens=False,
+    streamer=streamer
+)
+
+```
+
+## Examples
+
+`Goku-8x22B-v0.1` has been tested in generating text, answering questions based on long context, coding, and some reasoning. In the next version I will use more `math` and `coding` related datasets.
+
 This is a sample story written by [MaziyarPanahi/Goku-8x22B-v0.1](https://huggingface.co/MaziyarPanahi/Goku-8x22B-v0.1/)
 
+
 ```
 Goku had heard a commotion from his house but when he went to check he saw nothing. He thought to himself, "I'll let it go, it was probably just a bird or something. I'm sure it will be fine." But that was when he heard the commotion again, so he went outside and this time he saw two figures on the horizon. One of the figures was a giant pinkish-purple creature, while the other was small, pink, ball-shaped thing.