Quick test tune overtop of meta-llama/Llama-3.2-3B-Instruct using a ~50/50 mix of instruct and completion data.
Note: Training nowhere near complete so I'm unsure how strong of an effect it had. Still refuses requests like meta-llama/Llama-3.2-3B-Instruct.
Open LLM Leaderboard Evaluation Results
Detailed results can be found here
| Metric | Value |
|---|---|
| Avg. | 22.41 |
| IFEval (0-Shot) | 69.31 |
| BBH (3-Shot) | 23.81 |
| MATH Lvl 5 (4-Shot) | 10.42 |
| GPQA (0-shot) | 3.24 |
| MuSR (0-shot) | 4.05 |
| MMLU-PRO (5-shot) | 23.64 |
- Downloads last month
- 287
Model tree for PJMixers-Dev/LLaMa-3.2-Instruct-JankMix-v0.1-SFT-3B
Base model
meta-llama/Llama-3.2-3B-InstructEvaluation results
- strict accuracy on IFEval (0-Shot)Open LLM Leaderboard69.310
- normalized accuracy on BBH (3-Shot)Open LLM Leaderboard23.810
- exact match on MATH Lvl 5 (4-Shot)Open LLM Leaderboard10.420
- acc_norm on GPQA (0-shot)Open LLM Leaderboard3.240
- acc_norm on MuSR (0-shot)Open LLM Leaderboard4.050
- accuracy on MMLU-PRO (5-shot)test set Open LLM Leaderboard23.640