| license: llama2 | |
| tags: | |
| - code | |
| This is a quantized version of **WizardLM/WizardCoder-Python-7B-V1.0**, quantized using [ctranslate2](https://github.com/OpenNMT/CTranslate2) (see inference instructions there). | |
| **The license/caveats/intended usage is the same as the original model**. | |
| The quality of its output may have | |
| been negatively affected by the quantization process. | |
| The command run to quantize the model was: | |
| `ct2-transformers-converter --model ./models-hf/WizardLM/WizardCoder-Python-7B-V1.0 --quantization float16 --output_dir ./models-ct/WizardLM/WizardCoder-Python-7B-V1.0-ct2-float16` | |
| The quantization was run on a 'high-mem', CPU only (8 core, 51GB) colab instance and took approximately 10 minutes. | |