RWKV v6
The RWKV-6 (Parallelizable RNN with Transformer-level LLM Performance 6 pronounced as RwaKuv 6) implementation in llama.cpp finally seams stable so let's quantize some of the popular models using this archidecture:
- https://huggingface.co/RWKV/v6-Finch-14B-HF
- https://huggingface.co/RWKV/v6-Finch-7B-HF
- https://huggingface.co/RWKV/v6-Finch-3B-HF
- https://huggingface.co/RWKV/v6-Finch-1B6-HF
- https://huggingface.co/RWKV/rwkv-6-world-7b
- https://huggingface.co/RWKV/rwkv-6-world-3b-v2.1
- https://huggingface.co/RWKV/rwkv-6-world-3b
- https://huggingface.co/RWKV/rwkv-6-world-1b6
blindly queued :)
I have at least rwkv-6-world-7b already, from 2 months ago. does "stable" also mean "broken back then"?
yeah, probably all rwkv-6-world ones.
i attempted all, but only the 7b seems to have passed back then
anyway, night :)
I have at least rwkv-6-world-7b already, from 2 months ago. does "stable" also mean "broken back then"?
The fix regarding the issue with BOS/EOS tokens was already merged 2 month ago. So no reason to requant them. If you already tried them all 2 month ago they are unlikely to work now. What error do you get for them?
assert (self.dir_model / "rwkv_vocab_v20230424.txt").is_file()
for rwkv-6-world-3b. I think the others succeeded this time, so something might have changed. I've pushed it to nico1, so in a few minutes, you can examine it if you wish. Maybe that is also why there is a "2.1" of it.
And holy shit is the model search on hf slow now (the one that simply does a substring search in the model names on the user page).