File size: 698 Bytes
3dfd592 aa0c21a 3dfd592 0ab1a21 3dfd592 0ab1a21 3dfd592 |
1 2 3 4 5 6 7 8 9 10 11 12 |
This is the first model which converts Qwen2.5-7B's checkpoint to RWKV-7 architecture.
It's trained in one server with 8xA800 for one day which might not be that versatile. It shows an acceptable performance to chat with you fluently.
The shortage is that this base model can't do math and related tasks. I'll add a more balanced data to improve that model's capability later.
Please refer the https://github.com/yynil/RWKVinLLAMA/blob/rwkv_7/tests/test_chat_cli.py about how to use it.
```python
python tests/test_chat_cli.py --config_file configs/qwen_7b.yaml --is_hybrid --num_gpus 1
```
You may need to change the configuration yaml to point the path to the checkpoint path you downloaded.
|