KaraKaraWitch commited on
Commit
df69736
·
verified ·
1 Parent(s): 0a7f8f0

quick spelling fix

Browse files
Files changed (1) hide show
  1. README.md +1 -1
README.md CHANGED
@@ -10,7 +10,7 @@ Linear models offer a promising approach to significantly reduce computational c
10
 
11
  As demonstrated with our Qwerky-72B-Preview and prior models such as QRWKV6-32B Instruct Preview, we have successfully converted Qwen 2.5 72B into a RWKV variant without requiring a pretrain on the base model or retraining the model from scratch. Enabling us to test and validate the more efficient RWKV Linear attention with a much smaller budget. Since our preview, we have continued to refine our technique and managed to improve the model over the preview model iteration.
12
 
13
- As with the previous models, the model's inherent knowledge and dataset training are inherited from its "parent" model. Consequently, unlike previous RWKV models trained on over 100+ languages, the QRWKV model is limited to approximately 30 languages supported by the Qwen line of models.
14
 
15
  You may find our details of the process from our previous release, find it [here](https://huggingface.co/recursal/QRWKV6-32B-Instruct-Preview-v0.1).
16
 
 
10
 
11
  As demonstrated with our Qwerky-72B-Preview and prior models such as QRWKV6-32B Instruct Preview, we have successfully converted Qwen 2.5 72B into a RWKV variant without requiring a pretrain on the base model or retraining the model from scratch. Enabling us to test and validate the more efficient RWKV Linear attention with a much smaller budget. Since our preview, we have continued to refine our technique and managed to improve the model over the preview model iteration.
12
 
13
+ As with our previous models, the model's inherent knowledge and dataset training are inherited from its "parent" model. Consequently, unlike previous RWKV models trained on over 100+ languages, the QRWKV model is limited to approximately 30 languages supported by the Qwen line of models.
14
 
15
  You may find our details of the process from our previous release, find it [here](https://huggingface.co/recursal/QRWKV6-32B-Instruct-Preview-v0.1).
16