File size: 924 Bytes
48b1a23 4ad08f7 48b1a23 4ad08f7 48b1a23 4ad08f7 |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 |
---
base_model: hpcgroup/hpc-coder-v2-1.3b
library_name: transformers
tags:
- code
- hpc
- parallel
- axonn
- llama-cpp
- gguf-my-repo
---
# HPC-Coder-v2-1.3b-Q8_0-GGUF
This is the [HPC-Coder-v2-6.7b](https://huggingface.co/hpcgroup/hpc-coder-v2-1.3b) model with 8 bit quantized weights in the GGUF format that can be used with llama.cpp.
Refer to the [original model card](https://huggingface.co/hpcgroup/hpc-coder-v2-1.3b) for more details on the model.
## Use with llama.cpp
See the [llama.cpp repo](https://github.com/ggerganov/llama.cpp) for installation instructions.
You can then use the model as:
```bash
llama-cli --hf-repo hpcgroup/hpc-coder-v2-1.3b-Q8_0-GGUF --hf-file hpc-coder-v2-1.3b-q8_0.gguf -r "Below is an instruction that describes a task. Write a response that appropriately completes the request.\n\n### Instruction:" --in-prefix "\n" --in-suffix "\n### Response:\n" -c 8096 -p "your prompt here"
``` |