File size: 313 Bytes
d2edf21 eb668cc d2edf21 eb668cc 2d6fe0f d2edf21 eb668cc d2edf21 |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 |
---
base_model:
- unsloth/gemma-3-12b-it
- NewEden/Gemma-LN-Lora
library_name: transformers
tags:
- mergekit
- merge
---
Hparams:
- LR: 1e-5
- Weight Decay: 0.02
- grad-clip: 0.2
- R64, A32, Rslora, 0.1 drop out, 16bit lora (qlora is cope.)
pretrained on a buncha LNs, books, etc.
Used for a base for Pascal.
|