|
--- |
|
license: cc-by-nc-4.0 |
|
tags: |
|
- not-for-all-audiences |
|
- nsfw |
|
--- |
|
<!-- description start --> |
|
Exllamav2 4.65bpw quantization of CausalLM-RP-34B from [NeverSleep](https://huggingface.co/NeverSleep/CausalLM-RP-34B), quantized with default calibration dataset. |
|
> [!IMPORTANT] |
|
>Fits in 24GB VRAM with 32k+ context. Make sure to enable 4-bit cache option or you'll run into OOM errors. |
|
|
|
--- |
|
## Original Card |
|
## Description |
|
|
|
This repo contains fp16 files of CausalLM-RP-34B, a finetuned model of the CausalLM-34B Beta on multiple RP datasets. |
|
|
|
<!-- description end --> |
|
<!-- description start --> |
|
## Model used |
|
|
|
- [CausalLM/34b-beta](https://huggingface.co/CausalLM/34b-beta) |
|
|
|
### Prompt template ChatML |
|
|
|
|
|
``` |
|
<|im_start|>system |
|
{system_prompt}<|im_end|> |
|
<|im_start|>user |
|
{prompt}<|im_end|> |
|
<|im_start|>assistant |
|
{output}<|im_end|> |
|
``` |
