|
--- |
|
license: gpl-3.0 |
|
metrics: |
|
- perplexity |
|
pipeline_tag: text-generation |
|
tags: |
|
- LLaMa |
|
- text-generation-inference |
|
- ggml |
|
language: |
|
- en |
|
- bg |
|
- ca |
|
- cs |
|
- da |
|
- de |
|
- es |
|
- fr |
|
- hr |
|
- hu |
|
- it |
|
- nl |
|
- pl |
|
- pt |
|
- ro |
|
- ru |
|
- sl |
|
- sr |
|
- sv |
|
- uk |
|
library_name: adapter-transformers |
|
--- |
|
|
|
LLaMa 65B converted to ggml via LLaMa.cpp, then quantized to 4bit. |
|
|
|
Note: If you previously used the q4_0 model before April 26th, 2023, you are using an outdated model. I suggest redownloading for a better experience. |
|
Check https://github.com/ggerganov/llama.cpp#quantization for details on the different quantization types. |
|
|
|
I recommend the following settings when running as a good starting point: ```main.exe -m ggml-LLaMa-65B-q4_0.bin -n -1 -t 42 -c 2048 --temp 0.4 --interactive-first --repeat_penalty 1.2 --color``` |
|
|
|
Be aware that LLaMa is a text generation model, not a conversational one, and as such you will have to prompt it differently than, for example, Vicuna or ChatGPT. |
