| Original model: https://huggingface.co/Doctor-Shotgun/Nous-Capybara-limarpv3-34B | |
| Using cleaned pippa.parquet as calibration dataset. | |
| 5bpw - Can run 10k context in ~23.3GB VRAM with 8bit-cache option. |
| Original model: https://huggingface.co/Doctor-Shotgun/Nous-Capybara-limarpv3-34B | |
| Using cleaned pippa.parquet as calibration dataset. | |
| 5bpw - Can run 10k context in ~23.3GB VRAM with 8bit-cache option. |