|
--- |
|
base_model: hpcgroup/hpc-coder-v2-1.3b |
|
library_name: transformers |
|
tags: |
|
- code |
|
- hpc |
|
- parallel |
|
- axonn |
|
- llama-cpp |
|
- gguf-my-repo |
|
--- |
|
|
|
# HPC-Coder-v2-1.3b-Q8_0-GGUF |
|
This is the [HPC-Coder-v2-6.7b](https://huggingface.co/hpcgroup/hpc-coder-v2-1.3b) model with 8 bit quantized weights in the GGUF format that can be used with llama.cpp. |
|
Refer to the [original model card](https://huggingface.co/hpcgroup/hpc-coder-v2-1.3b) for more details on the model. |
|
|
|
## Use with llama.cpp |
|
See the [llama.cpp repo](https://github.com/ggerganov/llama.cpp) for installation instructions. |
|
You can then use the model as: |
|
|
|
```bash |
|
llama-cli --hf-repo hpcgroup/hpc-coder-v2-1.3b-Q8_0-GGUF --hf-file hpc-coder-v2-1.3b-q8_0.gguf -r "Below is an instruction that describes a task. Write a response that appropriately completes the request.\n\n### Instruction:" --in-prefix "\n" --in-suffix "\n### Response:\n" -c 8096 -p "your prompt here" |
|
``` |
