Create README.md

README.md

@@ -0,0 +1,108 @@
+---
+license: llama3.2
+language:
+- en
+base_model:
+- meta-llama/Llama-3.2-1B-Instruct
+library_name: transformers
+---
+# Model Name: Orca_Mini_v9_1_Llama-3.2-1B-Instruct
+
+**Orca_Mini_v9_1_Llama-3.2-1B-Instruct is trained with various SFT Datasets on [Llama-3.2-1B-Instruct](https://huggingface.co/meta-llama/Llama-3.2-1B-Instruct)**
+
+<img src="https://huggingface.co/pankajmathur/orca_mini_v5_8b/resolve/main/orca_minis_small.jpeg" width="auto" />
+
+<strong>
+Passionate about Generative AI?, Let's Connect! <a href="https://www.linkedin.com/in/pankajam" target="_blank">https://www.linkedin.com/in/pankajam</a>
+</strong>
+
+<br>
+
+### NOTICE
+By providing proper credit and attribution, you are granted permission to use this model as a foundational base for further Full fine tuning, DPO, PPO or ORPO tuning and any kind of Merges. 
+I actively encourage users to customize and enhance the model according to their specific needs, as this version is designed to be a comprehensive general model. 
+Dive in and innovate!
+
+
+### Example Usage
+Here is the Llama3 prompt format
+```
+<|begin_of_text|><|start_header_id|>system<|end_header_id|>
+You are Orca Mini, a helpful AI assistant.<|eot_id|>
+<|start_header_id|>user<|end_header_id|>
+Hello Orca Mini, what can you do for me?<|eot_id|>
+<|start_header_id|>assistant<|end_header_id|>
+```
+
+Below shows a code example on how to use this model in default half precision (bfloat16) format
+
+```python
+import torch
+from transformers import pipeline
+
+model_slug = "pankajmathur/orca_mini_v9_1_1B-Instruct"
+pipeline = pipeline(
+    "text-generation",
+    model=model_slug,
+    device_map="auto",
+)
+messages = [
+    {"role": "system", "content": "You are Orca Mini, a helpful AI assistant."},
+    {"role": "user", "content": "Hello Orca Mini, what can you do for me?"}
+]
+outputs = pipeline(messages, max_new_tokens=128, do_sample=True, temperature=0.01, top_k=100, top_p=0.95)
+print(outputs[0]["generated_text"][-1])
+```
+
+Below shows a code example on how to use this model in 4-bit format via bitsandbytes library
+
+```python
+import torch
+from transformers import BitsAndBytesConfig, pipeline
+
+model_slug = "pankajmathur/orca_mini_v9_1_1B-Instruct"
+quantization_config = BitsAndBytesConfig(
+    load_in_4bit=True,
+    bnb_4bit_quant_type="nf4",
+    bnb_4bit_compute_dtype="float16",
+    bnb_4bit_use_double_quant=True,
+)
+pipeline = pipeline(
+    "text-generation",
+    model=model_slug,
+    model_kwargs={"quantization_config": quantization_config},
+    device_map="auto",
+)
+messages = [
+    {"role": "system", "content": "You are Orca Mini, a helpful AI assistant."},
+    {"role": "user", "content": "Hello Orca Mini, what can you do for me?"}
+]
+outputs = pipeline(messages, max_new_tokens=128, do_sample=True, temperature=0.01, top_k=100, top_p=0.95)
+print(outputs[0]["generated_text"][-1])
+
+```
+
+Below shows a code example on how to use this model in 8-bit format via bitsandbytes library
+
+```python
+import torch
+from transformers import BitsAndBytesConfig, pipeline
+
+model_slug = "pankajmathur/orca_mini_v9_1_1B-Instruct"
+quantization_config = BitsAndBytesConfig(
+    load_in_8bit=True
+)
+pipeline = pipeline(
+    "text-generation",
+    model=model_slug,
+    model_kwargs={"quantization_config": quantization_config},
+    device_map="auto",
+)
+messages = [
+    {"role": "system", "content": "You are Orca Mini, a helpful AI assistant."},
+    {"role": "user", "content": "Hello Orca Mini, what can you do for me?"}
+]
+outputs = pipeline(messages, max_new_tokens=128, do_sample=True, temperature=0.01, top_k=100, top_p=0.95)
+print(outputs[0]["generated_text"][-1])
+
+```