CISCai commited on
Commit
d1f6790
·
verified ·
1 Parent(s): 2176a8c

Marked <|im_start|> as a special token, fixing tokenization.

Browse files
README.md CHANGED
@@ -24,6 +24,8 @@ This repo contains State Of The Art quantized GGUF format model files for [Yi-Co
24
 
25
  Quantization was done with an importance matrix that was trained for ~1M tokens (256 batches of 4096 tokens) of answers from the [CodeFeedback-Filtered-Instruction](https://huggingface.co/datasets/m-a-p/CodeFeedback-Filtered-Instruction) dataset.
26
 
 
 
27
  Corrected EOS (<|im_end|>) and added EOT (<|endoftext|>) token to prevent infinite responses (am I the only one actually dog-fooding my own quants?).
28
 
29
  Fill-in-Middle token metadata has been added, see [example](#simple-llama-cpp-python-example-fill-in-middle-code). NOTE: Yi's FIM requires support for [SPM infill mode](https://github.com/abetlen/llama-cpp-python/pull/1492)! However it seems it has not been extensively trained for this (perhaps not at all), so don't expect particularly great results...
 
24
 
25
  Quantization was done with an importance matrix that was trained for ~1M tokens (256 batches of 4096 tokens) of answers from the [CodeFeedback-Filtered-Instruction](https://huggingface.co/datasets/m-a-p/CodeFeedback-Filtered-Instruction) dataset.
26
 
27
+ **Update September 5th**: Marked <|im_start|> as a special token, fixing tokenization.
28
+
29
  Corrected EOS (<|im_end|>) and added EOT (<|endoftext|>) token to prevent infinite responses (am I the only one actually dog-fooding my own quants?).
30
 
31
  Fill-in-Middle token metadata has been added, see [example](#simple-llama-cpp-python-example-fill-in-middle-code). NOTE: Yi's FIM requires support for [SPM infill mode](https://github.com/abetlen/llama-cpp-python/pull/1492)! However it seems it has not been extensively trained for this (perhaps not at all), so don't expect particularly great results...
Yi-Coder-9B-Chat-bf16-00001-of-00002.gguf CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:573c1e10e24d71542d30c75f666f53a93e1baac20d7667cdeece728bbfb8b8b9
3
  size 1478133
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:c1f9006b2b3fabab9f796b643fc488ee6c4ead7522768c76e69d2761ff2ee3c3
3
  size 1478133
Yi-Coder-9B-Chat.IQ1_M.gguf CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:b78d928608b990ce33db54e7bc3ccc2a0cd8d40d2e9377ac0960eff418bb9f4f
3
  size 2181641152
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:6ac4ada21f1e92506dfa0c63d4b77d488ebed479c6d8e21ca1ed767101cb37ee
3
  size 2181641152
Yi-Coder-9B-Chat.IQ1_S.gguf CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:85936d853fa60593df331adb589a87c402ea7fa5f933025a393b3c294b357b34
3
  size 2014573504
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:36b3f5ce2f04d458a72c98819569873ac08ad28017034868050b368070207e80
3
  size 2014573504
Yi-Coder-9B-Chat.IQ2_M.gguf CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:14343048726c07cd0bd77afadb5005b2a65979129f6030af8ef0d1eb76051ba8
3
  size 3098112960
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:12203a97c90c2464a78fe97aca2d8f9b371ed08d374da72456f33de0d3635dda
3
  size 3098112960
Yi-Coder-9B-Chat.IQ2_S.gguf CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:c689d7fa1ebd6e2c142befbce70d457245335c085b082ba872b493caefad69db
3
  size 2875356096
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:1e8d3345865f550163495d1c1296d0f984bcafab82042f4bb0b4cf5c4df908dc
3
  size 2875356096
Yi-Coder-9B-Chat.IQ2_XS.gguf CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:02bbac077b8b7dce2e44b9de3db8f76d892aa7d3364d910e502e9093a183d89b
3
  size 2708009920
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:14f905b46f88737326eeecb13441f471e01d58b972eab1ece759186017339c52
3
  size 2708009920
Yi-Coder-9B-Chat.IQ2_XXS.gguf CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:888f6e659a9a28b4752c94f03b77602fb0a400b36084fba768eb89b8196e2f09
3
  size 2460087232
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:f8b1337522073c78a4b90692912b61930d2c5f0bcc6670f5d95c2a209ce79c2e
3
  size 2460087232
Yi-Coder-9B-Chat.IQ3_M.gguf CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:0588f4e66af2cfd0dc778a885d7684cf340d52641255f2bcf328be34059b56eb
3
  size 4055462848
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:acf6c496ce02c15ed057f7f3309d6c6ced33268c89dc6b9affbe961f80ec8a4b
3
  size 4055462848
Yi-Coder-9B-Chat.IQ3_S.gguf CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:551872b6ea7c83a9251b8cff65e36019b3d7ed3ec5f6d02535fc8b497dedb0ed
3
  size 3912577984
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:9e7b5d91cbc6672ef853eac657bd0b5878c935bdf4a9b4d0ff1fe9ed515a283e
3
  size 3912577984
Yi-Coder-9B-Chat.IQ3_XS.gguf CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:490a0fb256cfbeee1657dc01b37edee880c3e0d97a75f25af53cd0a8637a806b
3
  size 3717936064
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:7a3d2ae4e215f3f316c5fc9b93bc795657d87df542dddbd98299a1fe2f6b2e6c
3
  size 3717936064
Yi-Coder-9B-Chat.IQ3_XXS.gguf CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:7eb5101bc603af9e6bf1dc6c5c3bdb465af4652ab3506d405fbfe6def1b2b3a2
3
  size 3474322368
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:f4017cf9b95ec59351acd8ba840af6dc30dfff43d6212b956916f94f8b8dc271
3
  size 3474322368
Yi-Coder-9B-Chat.IQ4_XS.gguf CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:ed52436d62c1f25250b1914dd73ea4e8db247ed2752be0245aee6d412133b73d
3
  size 4785009600
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:57a7ef81c547163c623e97996d0e7e6c9f6196cb7878c6d04c1a6b47cb006e8a
3
  size 4785009600