Initial Commit

.gitattributes

@@ -33,3 +33,4 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
 *.zip filter=lfs diff=lfs merge=lfs -text
 *.zst filter=lfs diff=lfs merge=lfs -text
 *tfevents* filter=lfs diff=lfs merge=lfs -text
+*.gguf filter=lfs diff=lfs merge=lfs -text

README.md

@@ -1,3 +1,27 @@
 ---
 license: mit
 ---
+Original source of the GGUF file: [juanjgit's Orca Mini 3B GGUF Model Card](https://huggingface.co/juanjgit/orca_mini_3B-GGUF])
+
+Original model: [Pankaj Mathur's Orca Mini 3B.](https://huggingface.co/psmathur/orca_mini_3b)
+
+# Prompt
+```
+prompt = f"### System:\n{system}\n\n### User:\n{instruction}\n\n### Response:\n"
+```
+
+# Usage
+```
+from ctransformers import AutoModelForCausalLM
+
+llm = AutoModelForCausalLM.from_pretrained(
+        "juanjgit/orca_mini_3B-GGUF", model_file="orca-mini-3b.q4_0.gguf"
+    )
+
+system = "You are an AI assistant that follows instruction extremely well. Help as much as you can. Give short answers."
+instruction = "Which is the biggest city of India?"
+
+prompt = f"### System:\n{system}\n\n### User:\n{instruction}\n\n### Response:\n"
+for i in llm(prompt, stream=True):
+    print(i)
+```

orca-mini-3b.q4_0.gguf

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:8d07551d23240886d39d031c8785c57c4dd185b0f39b2e7f01d58f0f6c2404a8
+size 1928648352