Hugging Face's logo

nm-testing
/
TinyLlama-1.1B-compressed-tensors-kv-cache-scheme

Text Generation
Transformers
Safetensors
llama
text-generation-inference
Inference Endpoints
compressed-tensors
Model card Files Community
dtransposed commited on
Commit
dfc9eed
·
verified ·
1 Parent(s): 941e856

Upload config.json with huggingface_hub

Browse files
Files changed (1) hide show
  1. config.json +2 -2
config.json CHANGED
@@ -16,7 +16,7 @@
16
  "num_bits": 8,
17
  "observer": "minmax",
18
  "observer_kwargs": {},
19
- "strategy": "tensor",
20
  "symmetric": true,
21
  "type": "int"
22
  },
@@ -31,7 +31,7 @@
31
  "num_bits": 8,
32
  "observer": "minmax",
33
  "observer_kwargs": {},
34
- "strategy": "tensor",
35
  "symmetric": true,
36
  "type": "int"
37
  }
 
16
  "num_bits": 8,
17
  "observer": "minmax",
18
  "observer_kwargs": {},
19
+ "strategy": "channel",
20
  "symmetric": true,
21
  "type": "int"
22
  },
 
31
  "num_bits": 8,
32
  "observer": "minmax",
33
  "observer_kwargs": {},
34
+ "strategy": "channel",
35
  "symmetric": true,
36
  "type": "int"
37
  }