horheynm
/

Phi-3-mini-4k-instruct-kv_cache

compressed-tensors

Model card Files Files and versions Community

Phi-3-mini-4k-instruct-kv_cache / recipe.yaml

horheynm's picture

Upload folder using huggingface_hub

1ceab7e verified about 2 months ago

143 Bytes

	quant_stage:
	quant_modifiers:
	QuantizationModifier:
	kv_cache_scheme: {num_bits: 8, type: float, symmetric: true, strategy: tensor}