Hugging Face's logo

hgloow
/
Merged-AGI-7B-EXL2

Text Generation
Transformers
Safetensors
English
mistral
Math
exl2
text-generation-inference
Inference Endpoints
Model card Files Community
hgloow commited on
Commit
bbe25c0
·
1 Parent(s): 169a850

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +1 -1
README.md CHANGED
@@ -31,7 +31,7 @@ Zipped Quantization (if you want to download a single file)
31
  [wikitext-103-v1](https://huggingface.co/datasets/wikitext/blob/9a9e482b5987f9d25b3a9b2883fc6cc9fd8071b3/wikitext-103-v1/wikitext-test.parquet)
32
 
33
  ## Memory Usage
34
- Measured using ExLlamaV2 and 4096 max_seq_len with [Oobabooga's Text Generation WebUI](https://github.com/oobabooga/text-generation-webui/tree/main).
35
  | Branch | BPW | VRAM Usage | Description |
36
  | ------ | --- | ---------- | ----------- |
37
  [3.0bpw](https://huggingface.co/hgloow/Merged-AGI-7B-EXL2/tree/3.0bpw)|3.0|3.7 GB|For >=6GB VRAM cards
 
31
  [wikitext-103-v1](https://huggingface.co/datasets/wikitext/blob/9a9e482b5987f9d25b3a9b2883fc6cc9fd8071b3/wikitext-103-v1/wikitext-test.parquet)
32
 
33
  ## Memory Usage
34
+ Measured using ExLlamav2_HF and 4096 max_seq_len with [Oobabooga's Text Generation WebUI](https://github.com/oobabooga/text-generation-webui/tree/main).
35
  | Branch | BPW | VRAM Usage | Description |
36
  | ------ | --- | ---------- | ----------- |
37
  [3.0bpw](https://huggingface.co/hgloow/Merged-AGI-7B-EXL2/tree/3.0bpw)|3.0|3.7 GB|For >=6GB VRAM cards