GPTQ-for-SantaCoder
Visit GPTQ-for-SantaCoder for instructions on how to use the model weights here. If you want 4-bit weights, visit santacoder-GPTQ-4bit-128g.
Results
| SantaCoder | Bits | group-size | memory(MiB) | wikitext2 | ptb | c4 | stack | checkpoint size(MB) |
|---|---|---|---|---|---|---|---|---|
| FP32 | 32 | - | 4344.722 | 24.927 | 38.574 | 27.779 | 2.619 | 4394 |
| BF16 | 16 | - | 2173.680 | 24.960 | 38.597 | 27.794 | 2.621 | 2195 |
| GPTQ | 8 | -1 | 1396.548 | 24.936 | 38.592 | 27.785 | 2.619 | 1411 |
| GPTQ | 4 | -1 | 911.384 | 26.581 | 40.717 | 29.232 | 2.658 | 913 |
| GPTQ | 3 | -1 | - | 11761.473 | 7273.338 | 9124.941 | 2485.844 | 789 |
| GPTQ | 2 | -1 | - | 67976.797 | 68994.484 | 73294.438 | 45370.488 | 649 |
License
The model is licenses under the CodeML Open RAIL-M v0.1 license. You can find the full license here.
Acknowledgements
Thanks to everyone in BigCode who worked so hard to create these code models.
Inference Providers
NEW
This model is not currently available via any of the supported Inference Providers.
The model cannot be deployed to the HF Inference API:
The model has no library tag.