| This is the first model which converts Qwen2.5-7B's checkpoint to RWKV-7 architecture. | |
| It's trained in one server with 8xA800 for one day which might not be that versatile. It shows an acceptable performance to chat with you fluently. | |
| The shortage is that this base model can't do math and related tasks. I'll add a more balanced data to improve that model's capability later. | |
| Please refer the https://github.com/yynil/RWKVinLLAMA/blob/rwkv_7/tests/test_chat_cli.py about how to use it. | |
| ```python | |
| python tests/test_chat_cli.py --config_file configs/qwen_7b.yaml --is_hybrid --num_gpus 1 | |
| ``` | |
| You may need to change the configuration yaml to point the path to the checkpoint path you downloaded. | |