Update README.md

README.md

@@ -19,13 +19,8 @@ datasets:
 <img src="Damysus.png" width="500" alt="Damysus - the fastest giant"/>
 
 <!-- Provide a quick summary of what the model is/does. -->
-This model is a instruction-tuned version of Phi-2, a Transformer model with 2.7 billion parameters from Microsoft. 
-The model has undergone further training to better follow specific user instructions, enhancing its ability to perform tasks as directed and improve its interaction with users. 
-This additional training helps the model to understand context better, generate more accurate and relevant responses, and adapt to a wide range of language-based tasks such as:
-- Questions and Answers,
-- Data Extraction,
-- Structured Outputs (i.e., JSON outputs),
-- And providing explanations,
+This model is a GGUF version of [Damysus-2.7B-Chat](https://huggingface.co/prince-canuma/Damysus-2.7B-Chat).
+
 
 ## Model Description
 
@@ -56,99 +51,80 @@ This model inherits some of the base model's limitations, such as:
 - Limited Scope for code: Majority of Phi-2 training data is based in Python and use common packages such as "typing, math, random, collections, datetime, itertools". If the model generates Python scripts that utilize other packages or scripts in other languages, we strongly recommend users manually verify all API uses.
 - Language Limitations: The model is primarily designed to understand standard English. Informal English, slang, or any other languages might pose challenges to its comprehension, leading to potential misinterpretations or errors in response.
 
-## How to Get Started with the Model
+### On the command line, including multiple files at once
 
-Use the code below to get started with the model.
+I recommend using the `huggingface-hub` Python library:
 
-```python
-from transformers import pipeline, Conversation
+```shell
+pip3 install huggingface-hub
+```
 
-chatbot = pipeline("conversational", model="prince-canuma/Damysus-2.7B-Chat")
-conversation = Conversation("I'm looking for a movie - what's your favourite one?")
-output = chatbot(conversation)
+Then you can download any individual model file to the current directory, at high speed, with a command like this:
 
-print(output)
+```shell
+huggingface-cli download prince-canuma/Damysus-2.7B-Chat-GGUF Damysus-2.7B-Chat.Q4_K_M.gguf --local-dir . --local-dir-use-symlinks False
 ```
 
-Or you can instatiate the model and tokenizer directly
-```python
-from transformers import AutoTokenizer, AutoModelForCausalLM
+<details>
+  <summary>More advanced huggingface-cli download usage (click to read)</summary>
 
-tokenizer = AutoTokenizer.from_pretrained("prince-canuma/Damysus-2.7B-Chat")
-model = AutoModelForCausalLM.from_pretrained("prince-canuma/Damysus-2.7B-Chat")
+You can also download multiple files at once with a pattern:
 
-inputs = tokenizer.apply_chat_template(
-    [
-        {"content":"You are an helpful AI assistant","role":"system"},
-        {"content":"I'm looking for a movie - what's your favourite one?","role":"user"},
-    ], add_generation_prompt=True, return_tensors="pt",
-).to("cuda")
+```shell
+huggingface-cli download prince-canuma/Damysus-2.7B-Chat-GGUF --local-dir . --local-dir-use-symlinks False --include='*Q4_K*gguf'
+```
 
-outputs = model.generate(inputs, do_sample=False, max_new_tokens=256)
+For more documentation on downloading with `huggingface-cli`, please see: [HF -> Hub Python Library -> Download files -> Download from the CLI](https://huggingface.co/docs/huggingface_hub/guides/download#download-from-the-cli).
 
-input_length = inputs.shape[1]
-print(tokenizer.batch_decode(outputs[:, input_length:], skip_special_tokens=True)[0])
+To accelerate downloads on fast connections (1Gbit/s or higher), install `hf_transfer`:
+
+```shell
+pip3 install hf_transfer
 ```
 
-Output:
+And set environment variable `HF_HUB_ENABLE_HF_TRANSFER` to `1`:
+
 ```shell
-My favorite movie is "The Shawshank Redemption."
+HF_HUB_ENABLE_HF_TRANSFER=1 huggingface-cli download prince-canuma/Damysus-2.7B-Chat-GGUF Damysus-2.7B-Chat.Q4_K_M.gguf --local-dir . --local-dir-use-symlinks False
+```
+
+Windows Command Line users: You can set the environment variable by running `set HF_HUB_ENABLE_HF_TRANSFER=1` before the download command.
+</details>
+<!-- README_GGUF.md-how-to-download end -->
 
-It's a powerful and inspiring story about hope, friendship, and redemption.
-The performances by Tim Robbins and Morgan Freeman are exceptional,
-and the film's themes and messages are timeless.
+<!-- README_GGUF.md-how-to-run start -->
+## Example `llama.cpp` command
 
-I highly recommend it to anyone who enjoys a well-crafted and emotionally engaging story.
+Make sure you are using `llama.cpp` from commit [d0cee0d](https://github.com/ggerganov/llama.cpp/commit/d0cee0d36d5be95a0d9088b674dbb27354107221) or later.
+
+```shell
+!./main -m ../Damysus-2.7B-Chat-GGUF/Damysus-2.7B-Chat.Q4_K_M.gguf \
+  --color -c 2048 --temp 0 \
+  --prompt "<|im_start|>system\nYou are a helpful assistant. Please keep your answers short.<|im_end|>\n<|im_start|>user\nCount to ten<|im_end|>\n" \
+  -n 256 --in-suffix "<|im_start|>assistant\n" -r "User:" -e --verbose-prompt
 ```
 
-### Structured Output
-```python
-from transformers import AutoTokenizer, AutoModelForCausalLM
-
-tokenizer = AutoTokenizer.from_pretrained("prince-canuma/Damysus-2.7B-Chat")
-model = AutoModelForCausalLM.from_pretrained("prince-canuma/Damysus-2.7B-Chat")
-
-inputs = tokenizer.apply_chat_template(
-    [
-        {"content":"You are a Robot that ONLY outputs JSON. Use this structure: {'entities': [{'type':..., 'name':...}]}.","role":"system"},
-        {"content":""""Extract the entities of type 'technology' and 'file_type' in JSON format from the following passage: AI is a transformative
-                      force in document processing employing technologies such as 'Machine Learning (ML), Natural Language Processing (NLP) and
-                      Optical Character Recognition (OCR) to understand, interpret, and summarize text. These technologies enhance accuracy,
-                      increase efficiency, and allow you and your company to process high volumes of data in short amount of time.
-                      For instance, you can easily extract key points and summarize a large PDF document (i.e., 500 pages) in just a few seconds.""",
-        "role":"user"},
-    ], add_generation_prompt=True, return_tensors="pt",
-).to("cuda")
-
-outputs = model.generate(inputs, do_sample=False, max_new_tokens=256)
-
-input_length = inputs.shape[1]
-print(tokenizer.batch_decode(outputs[:, input_length:], skip_special_tokens=True)[0])
+or 
+
+```shell
+!./main -m ../Damysus-2.7B-Chat-GGUF/Damysus-2.7B-Chat.Q4_K_M.gguf \
+  --color -c 2048 --temp 0 \
+  -p "You are a helpful assistant. Please keep your answers short." -n 256 --in-suffix "<|im_start|>assistant\n" \
+  -r "User:" -e --verbose-prompt -cml
+
 ```
+- `-ngl N` offload N number of layers to GPU. Remove it if you don't have GPU acceleration.
 
-Output:
-```json
-{
-  "entities": [
-    {
-      "type": "technology",
-      "name": "Machine Learning (ML)"
-    },
-    {
-      "type": "technology",
-      "name": "Natural Language Processing (NLP)"
-    },
-    {
-      "type": "technology",
-      "name": "Optical Character Recognition (OCR)"
-    },
-    {
-      "type": "file_type",
-      "name": "PDF"
-    },
-  ]
-}
+- `-c 2048` set desired sequence length. For extended sequence models - eg 8K, 16K, 32K - the necessary RoPE scaling parameters are read from the GGUF file and set by llama.cpp automatically. Note that longer sequence lengths require much more resources, so you may need to reduce this value.
+
+- Add `-i -ins` or `-cml` argument for interactive chat-style conversation.
+
+For other parameters and how to use them, please refer to [the llama.cpp documentation](https://github.com/ggerganov/llama.cpp/blob/master/examples/main/README.md) or run:
+
+```shell
+!./main --help
 ```
+
 ## Training Details
 
 ### Training Data