Quantized models

.gitattributes

@@ -33,3 +33,9 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
 *.zip filter=lfs diff=lfs merge=lfs -text
 *.zst filter=lfs diff=lfs merge=lfs -text
 *tfevents* filter=lfs diff=lfs merge=lfs -text
+deepseek-coder-7b-instruct-v1.5.f16.gguf filter=lfs diff=lfs merge=lfs -text
+deepseek-coder-7b-instruct-v1.5.q4_0.gguf filter=lfs diff=lfs merge=lfs -text
+deepseek-coder-7b-instruct-v1.5.q4_k.gguf filter=lfs diff=lfs merge=lfs -text
+deepseek-coder-7b-instruct-v1.5.q5_k.gguf filter=lfs diff=lfs merge=lfs -text
+deepseek-coder-7b-instruct-v1.5.q6_k.gguf filter=lfs diff=lfs merge=lfs -text
+deepseek-coder-7b-instruct-v1.5.q8_0.gguf filter=lfs diff=lfs merge=lfs -text

Modelfile

@@ -0,0 +1,7 @@
+FROM model.gguf
+TEMPLATE """{{ .System }}
+### Instruction:
+{{ .Prompt }}
+### Response:
+"""
+SYSTEM """You are an AI programming assistant, utilizing the Deepseek Coder model, developed by Deepseek Company, and you only answer questions related to computer science. For politically sensitive questions, security and privacy issues, and other non-computer science questions, you will refuse to answer."""

README.md

@@ -0,0 +1,55 @@
+---
+license: other
+license_name: deepseek
+license_link: LICENSE
+---
+
+<p align="center">
+<img width="1000px" alt="DeepSeek Coder" src="https://github.com/deepseek-ai/DeepSeek-Coder/blob/main/pictures/logo.png?raw=true">
+</p>
+<p align="center"><a href="https://www.deepseek.com/">[🏠Homepage]</a>  |  <a href="https://coder.deepseek.com/">[🤖 Chat with DeepSeek Coder]</a>  |  <a href="https://discord.gg/Tc7c45Zzu5">[Discord]</a>  |  <a href="https://github.com/guoday/assert/blob/main/QR.png?raw=true">[Wechat(微信)]</a> </p>
+<hr>
+
+
+
+
+### 1. Introduction of Deepseek-Coder-7B-Instruct v1.5
+
+Deepseek-Coder-7B-Instruct-v1.5 is continue pre-trained from Deepseek-LLM 7B on 2T tokens by employing a window size of 4K and next token prediction objective, and then fine-tuned on 2B tokens of instruction data.
+
+- **Home Page:** [DeepSeek](https://deepseek.com/)
+- **Repository:** [deepseek-ai/deepseek-coder](https://github.com/deepseek-ai/deepseek-coder)
+- **Chat With DeepSeek Coder:** [DeepSeek-Coder](https://coder.deepseek.com/)
+
+
+
+### 2. Evaluation Results
+<img width="1000px" alt="DeepSeek Coder" src="https://cdn-uploads.huggingface.co/production/uploads/6538815d1bdb3c40db94fbfa/xOtCTW5xdoLCKY4FR6tri.png">
+
+
+
+### 3. How to Use
+Here give some examples of how to use our model.
+#### Chat Model Inference
+```python
+from transformers import AutoTokenizer, AutoModelForCausalLM
+tokenizer = AutoTokenizer.from_pretrained("deepseek-ai/deepseek-coder-7b-instruct-v1.5", trust_remote_code=True)
+model = AutoModelForCausalLM.from_pretrained("deepseek-ai/deepseek-coder-7b-instruct-v1.5", trust_remote_code=True).cuda()
+messages=[
+    { 'role': 'user', 'content': "write a quick sort algorithm in python."}
+]
+inputs = tokenizer.apply_chat_template(messages, add_generation_prompt=True, return_tensors="pt").to(model.device)
+
+outputs = model.generate(inputs, max_new_tokens=512, do_sample=False, top_k=50, top_p=0.95, num_return_sequences=1, eos_token_id=tokenizer.eos_token_id)
+print(tokenizer.decode(outputs[0][len(inputs[0]):], skip_special_tokens=True))
+```
+
+### 4. License
+This code repository is licensed under the MIT License. The use of DeepSeek Coder models is subject to the Model License. DeepSeek Coder supports commercial use.
+
+See the [LICENSE-MODEL](https://github.com/deepseek-ai/deepseek-coder/blob/main/LICENSE-MODEL) for more details.
+
+### 5. Contact
+
+If you have any questions, please raise an issue or contact us at [[email protected]](mailto:[email protected]).
+

deepseek-coder-7b-instruct-v1.5.f16.gguf

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:f4399d415270a9da85dc4fbdcbd8c3ef378b5087ac8d3a9e0a8456410c076194
+size 13825630528

deepseek-coder-7b-instruct-v1.5.q4_0.gguf

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:70fd6d46febb4b08b76d63fd21ca79a65fd663fd4965316aa196e9754e946b55
+size 4000473440

deepseek-coder-7b-instruct-v1.5.q4_k.gguf

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:d76948555c437388dbb32efe09ccb1059b154202773ca3e9bbfa412fbab0e22e
+size 4223770976

deepseek-coder-7b-instruct-v1.5.q5_k.gguf

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:2f1b5ad7e7c48a83e8717852ab944f0c8580eb91c034ec1353ce6154bcf73c6b
+size 4926841184

deepseek-coder-7b-instruct-v1.5.q6_k.gguf

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:286c8d46182e933bf90e4ef4395f21850232a786fa752daa674aafc7bafd899c
+size 5673853280

deepseek-coder-7b-instruct-v1.5.q8_0.gguf

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:7836fd479d16ae9beebf38de444de1fb86d257499b3e6dd0bc77c4d1b16eb45c
+size 7347396960