training config
#4
by
ccdv
- opened
hi,
can you share the training config please? did you train using lora?
thank you
Hi,
This is what I used (pretty much), you just need to remove the FSDP part and the QLoRA.
https://github.com/axolotl-ai-cloud/axolotl/blob/main/examples/qwen2/qlora-fsdp.yaml
Since it's a pretty tiny model I did full fine-tunes of the base model.