Hugging Face's logo

altomek
/
RE-70B-AS3D

Text Generation
Transformers
Safetensors
English
llama
Merge
conversational
text-generation-inference
Inference Endpoints
Model card Files Community
altomek commited on
Commit
d43f711
·
verified ·
1 Parent(s): 914c8f7

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +1 -0
README.md CHANGED
@@ -42,6 +42,7 @@ Use Llama 3 template, model with ALpaca template sometimes halucinates and gener
42
  More quants are comming soon... and I need to redo GGUF quants(!) as they do not encode chat template in tokenizer_config... if your inference engine uses chat template from GGUF file you will see halucinations. However GGUFs work fine for me in SillyTawern & text-generation-webui combo...
43
 
44
  - [GGUF](https://huggingface.co/altomek/RE-70B-AS3D-GGUF) --> TO BE UPDATED!
 
45
  - [3.5 BPW](https://huggingface.co/altomek/RE-70B-AS3D-3.5bpw-EXL2)
46
  - [3.75 BPW](https://huggingface.co/altomek/RE-70B-AS3D-3.75bpw-EXL2)
47
  - [4 BPW](https://huggingface.co/altomek/RE-70B-AS3D-4bpw-EXL2)
 
42
  More quants are comming soon... and I need to redo GGUF quants(!) as they do not encode chat template in tokenizer_config... if your inference engine uses chat template from GGUF file you will see halucinations. However GGUFs work fine for me in SillyTawern & text-generation-webui combo...
43
 
44
  - [GGUF](https://huggingface.co/altomek/RE-70B-AS3D-GGUF) --> TO BE UPDATED!
45
+ - [3 BPW](https://huggingface.co/altomek/RE-70B-AS3D-3bpw-EXL2)
46
  - [3.5 BPW](https://huggingface.co/altomek/RE-70B-AS3D-3.5bpw-EXL2)
47
  - [3.75 BPW](https://huggingface.co/altomek/RE-70B-AS3D-3.75bpw-EXL2)
48
  - [4 BPW](https://huggingface.co/altomek/RE-70B-AS3D-4bpw-EXL2)