Marked <|im_start|> as a special token, fixing tokenization.
Browse files- README.md +2 -0
- Yi-Coder-9B-Chat-bf16-00001-of-00002.gguf +1 -1
- Yi-Coder-9B-Chat.IQ1_M.gguf +1 -1
- Yi-Coder-9B-Chat.IQ1_S.gguf +1 -1
- Yi-Coder-9B-Chat.IQ2_M.gguf +1 -1
- Yi-Coder-9B-Chat.IQ2_S.gguf +1 -1
- Yi-Coder-9B-Chat.IQ2_XS.gguf +1 -1
- Yi-Coder-9B-Chat.IQ2_XXS.gguf +1 -1
- Yi-Coder-9B-Chat.IQ3_M.gguf +1 -1
- Yi-Coder-9B-Chat.IQ3_S.gguf +1 -1
- Yi-Coder-9B-Chat.IQ3_XS.gguf +1 -1
- Yi-Coder-9B-Chat.IQ3_XXS.gguf +1 -1
- Yi-Coder-9B-Chat.IQ4_XS.gguf +1 -1
README.md
CHANGED
@@ -24,6 +24,8 @@ This repo contains State Of The Art quantized GGUF format model files for [Yi-Co
|
|
24 |
|
25 |
Quantization was done with an importance matrix that was trained for ~1M tokens (256 batches of 4096 tokens) of answers from the [CodeFeedback-Filtered-Instruction](https://huggingface.co/datasets/m-a-p/CodeFeedback-Filtered-Instruction) dataset.
|
26 |
|
|
|
|
|
27 |
Corrected EOS (<|im_end|>) and added EOT (<|endoftext|>) token to prevent infinite responses (am I the only one actually dog-fooding my own quants?).
|
28 |
|
29 |
Fill-in-Middle token metadata has been added, see [example](#simple-llama-cpp-python-example-fill-in-middle-code). NOTE: Yi's FIM requires support for [SPM infill mode](https://github.com/abetlen/llama-cpp-python/pull/1492)! However it seems it has not been extensively trained for this (perhaps not at all), so don't expect particularly great results...
|
|
|
24 |
|
25 |
Quantization was done with an importance matrix that was trained for ~1M tokens (256 batches of 4096 tokens) of answers from the [CodeFeedback-Filtered-Instruction](https://huggingface.co/datasets/m-a-p/CodeFeedback-Filtered-Instruction) dataset.
|
26 |
|
27 |
+
**Update September 5th**: Marked <|im_start|> as a special token, fixing tokenization.
|
28 |
+
|
29 |
Corrected EOS (<|im_end|>) and added EOT (<|endoftext|>) token to prevent infinite responses (am I the only one actually dog-fooding my own quants?).
|
30 |
|
31 |
Fill-in-Middle token metadata has been added, see [example](#simple-llama-cpp-python-example-fill-in-middle-code). NOTE: Yi's FIM requires support for [SPM infill mode](https://github.com/abetlen/llama-cpp-python/pull/1492)! However it seems it has not been extensively trained for this (perhaps not at all), so don't expect particularly great results...
|
Yi-Coder-9B-Chat-bf16-00001-of-00002.gguf
CHANGED
@@ -1,3 +1,3 @@
|
|
1 |
version https://git-lfs.github.com/spec/v1
|
2 |
-
oid sha256:
|
3 |
size 1478133
|
|
|
1 |
version https://git-lfs.github.com/spec/v1
|
2 |
+
oid sha256:c1f9006b2b3fabab9f796b643fc488ee6c4ead7522768c76e69d2761ff2ee3c3
|
3 |
size 1478133
|
Yi-Coder-9B-Chat.IQ1_M.gguf
CHANGED
@@ -1,3 +1,3 @@
|
|
1 |
version https://git-lfs.github.com/spec/v1
|
2 |
-
oid sha256:
|
3 |
size 2181641152
|
|
|
1 |
version https://git-lfs.github.com/spec/v1
|
2 |
+
oid sha256:6ac4ada21f1e92506dfa0c63d4b77d488ebed479c6d8e21ca1ed767101cb37ee
|
3 |
size 2181641152
|
Yi-Coder-9B-Chat.IQ1_S.gguf
CHANGED
@@ -1,3 +1,3 @@
|
|
1 |
version https://git-lfs.github.com/spec/v1
|
2 |
-
oid sha256:
|
3 |
size 2014573504
|
|
|
1 |
version https://git-lfs.github.com/spec/v1
|
2 |
+
oid sha256:36b3f5ce2f04d458a72c98819569873ac08ad28017034868050b368070207e80
|
3 |
size 2014573504
|
Yi-Coder-9B-Chat.IQ2_M.gguf
CHANGED
@@ -1,3 +1,3 @@
|
|
1 |
version https://git-lfs.github.com/spec/v1
|
2 |
-
oid sha256:
|
3 |
size 3098112960
|
|
|
1 |
version https://git-lfs.github.com/spec/v1
|
2 |
+
oid sha256:12203a97c90c2464a78fe97aca2d8f9b371ed08d374da72456f33de0d3635dda
|
3 |
size 3098112960
|
Yi-Coder-9B-Chat.IQ2_S.gguf
CHANGED
@@ -1,3 +1,3 @@
|
|
1 |
version https://git-lfs.github.com/spec/v1
|
2 |
-
oid sha256:
|
3 |
size 2875356096
|
|
|
1 |
version https://git-lfs.github.com/spec/v1
|
2 |
+
oid sha256:1e8d3345865f550163495d1c1296d0f984bcafab82042f4bb0b4cf5c4df908dc
|
3 |
size 2875356096
|
Yi-Coder-9B-Chat.IQ2_XS.gguf
CHANGED
@@ -1,3 +1,3 @@
|
|
1 |
version https://git-lfs.github.com/spec/v1
|
2 |
-
oid sha256:
|
3 |
size 2708009920
|
|
|
1 |
version https://git-lfs.github.com/spec/v1
|
2 |
+
oid sha256:14f905b46f88737326eeecb13441f471e01d58b972eab1ece759186017339c52
|
3 |
size 2708009920
|
Yi-Coder-9B-Chat.IQ2_XXS.gguf
CHANGED
@@ -1,3 +1,3 @@
|
|
1 |
version https://git-lfs.github.com/spec/v1
|
2 |
-
oid sha256:
|
3 |
size 2460087232
|
|
|
1 |
version https://git-lfs.github.com/spec/v1
|
2 |
+
oid sha256:f8b1337522073c78a4b90692912b61930d2c5f0bcc6670f5d95c2a209ce79c2e
|
3 |
size 2460087232
|
Yi-Coder-9B-Chat.IQ3_M.gguf
CHANGED
@@ -1,3 +1,3 @@
|
|
1 |
version https://git-lfs.github.com/spec/v1
|
2 |
-
oid sha256:
|
3 |
size 4055462848
|
|
|
1 |
version https://git-lfs.github.com/spec/v1
|
2 |
+
oid sha256:acf6c496ce02c15ed057f7f3309d6c6ced33268c89dc6b9affbe961f80ec8a4b
|
3 |
size 4055462848
|
Yi-Coder-9B-Chat.IQ3_S.gguf
CHANGED
@@ -1,3 +1,3 @@
|
|
1 |
version https://git-lfs.github.com/spec/v1
|
2 |
-
oid sha256:
|
3 |
size 3912577984
|
|
|
1 |
version https://git-lfs.github.com/spec/v1
|
2 |
+
oid sha256:9e7b5d91cbc6672ef853eac657bd0b5878c935bdf4a9b4d0ff1fe9ed515a283e
|
3 |
size 3912577984
|
Yi-Coder-9B-Chat.IQ3_XS.gguf
CHANGED
@@ -1,3 +1,3 @@
|
|
1 |
version https://git-lfs.github.com/spec/v1
|
2 |
-
oid sha256:
|
3 |
size 3717936064
|
|
|
1 |
version https://git-lfs.github.com/spec/v1
|
2 |
+
oid sha256:7a3d2ae4e215f3f316c5fc9b93bc795657d87df542dddbd98299a1fe2f6b2e6c
|
3 |
size 3717936064
|
Yi-Coder-9B-Chat.IQ3_XXS.gguf
CHANGED
@@ -1,3 +1,3 @@
|
|
1 |
version https://git-lfs.github.com/spec/v1
|
2 |
-
oid sha256:
|
3 |
size 3474322368
|
|
|
1 |
version https://git-lfs.github.com/spec/v1
|
2 |
+
oid sha256:f4017cf9b95ec59351acd8ba840af6dc30dfff43d6212b956916f94f8b8dc271
|
3 |
size 3474322368
|
Yi-Coder-9B-Chat.IQ4_XS.gguf
CHANGED
@@ -1,3 +1,3 @@
|
|
1 |
version https://git-lfs.github.com/spec/v1
|
2 |
-
oid sha256:
|
3 |
size 4785009600
|
|
|
1 |
version https://git-lfs.github.com/spec/v1
|
2 |
+
oid sha256:57a7ef81c547163c623e97996d0e7e6c9f6196cb7878c6d04c1a6b47cb006e8a
|
3 |
size 4785009600
|