| base_model: | |
| - unsloth/gemma-3-12b-it | |
| - NewEden/Gemma-LN-Lora | |
| library_name: transformers | |
| tags: | |
| - mergekit | |
| - merge | |
| Hparams: | |
| - LR: 1e-5 | |
| - Weight Decay: 0.02 | |
| - grad-clip: 0.2 | |
| - R64, A32, Rslora, 0.1 drop out, 16bit lora (qlora is cope.) | |
| pretrained on a buncha LNs, books, etc. | |
| Used for a base for Pascal. | |