bartowski commited on
Commit
289bc54
1 Parent(s): ad77b20

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +3 -3
README.md CHANGED
@@ -21,7 +21,7 @@ All quants made using imatrix option with dataset from [here](https://gist.githu
21
 
22
  User: {prompt}
23
 
24
- Assistant: <|end▁of▁sentence|>Assistant:
25
  ```
26
 
27
  ## Download a file (not the whole branch) from below:
@@ -29,9 +29,9 @@ Assistant: <|end▁of▁sentence|>Assistant:
29
  | Filename | Quant type | File Size | Description |
30
  | -------- | ---------- | --------- | ----------- |
31
  | [DeepSeek-Coder-V2-Instruct-Q4_K_M.gguf](https://huggingface.co/bartowski/DeepSeek-Coder-V2-Instruct-GGUF/tree/main/DeepSeek-Coder-V2-Instruct-Q4_K_M.gguf) | Q4_K_M | 142.45GB | Good quality, uses about 4.83 bits per weight, *recommended*. |
32
- | [DeepSeek-Coder-V2-Instruct-Q3_K_XL.gguf](https://huggingface.co/bartowski/DeepSeek-Coder-V2-Instruct-GGUF/tree/main/DeepSeek-Coder-V2-Instruct-Q3_K_XL.gguf) | Q3_K_XL | | *Experimental*, uses f16 for embed and output weights. Please provide any feedback of differences. Lower quality but usable, good for low RAM availability. |
33
  | [DeepSeek-Coder-V2-Instruct-Q3_K_M.gguf](https://huggingface.co/bartowski/DeepSeek-Coder-V2-Instruct-GGUF/tree/main/DeepSeek-Coder-V2-Instruct-Q3_K_M.gguf) | Q3_K_M | 112.7GB | Relatively low quality but usable. |
34
- | [DeepSeek-Coder-V2-Instruct-Q2_K_L.gguf](https://huggingface.co/bartowski/DeepSeek-Coder-V2-Instruct-GGUF/tree/main/DeepSeek-Coder-V2-Instruct-Q2_K_L.gguf) | Q2_K_L | | *Experimental*, uses f16 for embed and output weights. Please provide any feedback of differences. Low quality but usable. |
35
  | [DeepSeek-Coder-V2-Instruct-Q2_K.gguf](https://huggingface.co/bartowski/DeepSeek-Coder-V2-Instruct-GGUF/tree/main/DeepSeek-Coder-V2-Instruct-Q2_K.gguf) | Q2_K | 86.0GB | Low quality but usable. |
36
  | [DeepSeek-Coder-V2-Instruct-IQ2_XS.gguf](https://huggingface.co/bartowski/DeepSeek-Coder-V2-Instruct-GGUF/tree/main/DeepSeek-Coder-V2-Instruct-IQ2_XS.gguf) | IQ2_XS | 68.7GB | Lower quality, uses SOTA techniques to be usable. |
37
  | [DeepSeek-Coder-V2-Instruct-IQ1_M.gguf](https://huggingface.co/bartowski/DeepSeek-Coder-V2-Instruct-GGUF/tree/main/DeepSeek-Coder-V2-Instruct-IQ1_M.gguf) | IQ1_M | 52.7GB | Extremely low quality, *not* recommended. |
 
21
 
22
  User: {prompt}
23
 
24
+ Assistant: <|end▁of▁sentence|>
25
  ```
26
 
27
  ## Download a file (not the whole branch) from below:
 
29
  | Filename | Quant type | File Size | Description |
30
  | -------- | ---------- | --------- | ----------- |
31
  | [DeepSeek-Coder-V2-Instruct-Q4_K_M.gguf](https://huggingface.co/bartowski/DeepSeek-Coder-V2-Instruct-GGUF/tree/main/DeepSeek-Coder-V2-Instruct-Q4_K_M.gguf) | Q4_K_M | 142.45GB | Good quality, uses about 4.83 bits per weight, *recommended*. |
32
+ | [DeepSeek-Coder-V2-Instruct-Q3_K_XL.gguf](https://huggingface.co/bartowski/DeepSeek-Coder-V2-Instruct-GGUF/tree/main/DeepSeek-Coder-V2-Instruct-Q3_K_XL.gguf) | Q3_K_XL | 123.8GB | *Experimental*, uses f16 for embed and output weights. Please provide any feedback of differences. Lower quality but usable, good for low RAM availability. |
33
  | [DeepSeek-Coder-V2-Instruct-Q3_K_M.gguf](https://huggingface.co/bartowski/DeepSeek-Coder-V2-Instruct-GGUF/tree/main/DeepSeek-Coder-V2-Instruct-Q3_K_M.gguf) | Q3_K_M | 112.7GB | Relatively low quality but usable. |
34
+ | [DeepSeek-Coder-V2-Instruct-Q2_K_L.gguf](https://huggingface.co/bartowski/DeepSeek-Coder-V2-Instruct-GGUF/tree/main/DeepSeek-Coder-V2-Instruct-Q2_K_L.gguf) | Q2_K_L | 87.5GB | *Experimental*, uses f16 for embed and output weights. Please provide any feedback of differences. Low quality but usable. |
35
  | [DeepSeek-Coder-V2-Instruct-Q2_K.gguf](https://huggingface.co/bartowski/DeepSeek-Coder-V2-Instruct-GGUF/tree/main/DeepSeek-Coder-V2-Instruct-Q2_K.gguf) | Q2_K | 86.0GB | Low quality but usable. |
36
  | [DeepSeek-Coder-V2-Instruct-IQ2_XS.gguf](https://huggingface.co/bartowski/DeepSeek-Coder-V2-Instruct-GGUF/tree/main/DeepSeek-Coder-V2-Instruct-IQ2_XS.gguf) | IQ2_XS | 68.7GB | Lower quality, uses SOTA techniques to be usable. |
37
  | [DeepSeek-Coder-V2-Instruct-IQ1_M.gguf](https://huggingface.co/bartowski/DeepSeek-Coder-V2-Instruct-GGUF/tree/main/DeepSeek-Coder-V2-Instruct-IQ1_M.gguf) | IQ1_M | 52.7GB | Extremely low quality, *not* recommended. |