The gutenberg finetunes
So I've been testing a lot of mistral nemo 12b and gemma 2 9b finetunes, for creative writing and storytelling. SPPO, SimPO, and Nephillim v3 all did really well for Gemma 9b, with I think the SPPO finetune edging them all out slightly, and to my surprise your gutenberg 9b tune was on par or slightly better in my testing and small test results, maybe because it was tuned on top of the already very good SPPO model.
This brings me to this model, the 12b gutenberg finetune. While quite good, better than most of the mistral nemo 12b finetunes and merges I've tried (honestly a lot of them are just plain bad), there was still a few better or as good. Magnum mini 1.1 is a contender (not v2, the v2 magnum model is terrible), not sure which of the two is better between this one and the magnum mini 1.1, it's close, but there's one that even better. A slimpo nemo model that's slipped under the radar, romulus slimpo 12b. I highly recommend it, might be worth doing your gutenberg training over that instead of regular mistral nemo.
https://huggingface.co/axolotl-ai-co/romulus-mistral-nemo-12b-simpo
Nemo is tough to finetune; it seems very sensitive to overfitting.
I appreciate your feedback! I'll try out my Gutenberg finetune on Romulus :)
Nemo is tough to finetune; it seems very sensitive to overfitting.
I appreciate your feedback! I'll try out my Gutenberg finetune on Romulus :)
You may need to play with the training parameters then, maybe the learning rate, rank, alpha, etc, or even just do 2 epochs instead of two. Probably best to try a couple things and see which give the best results. Either way, good luck! There was also talk of a better version of DPO called, KTO. Not sure if it lives up to the hype but might be worth looking into.