Chryslerx10's picture
Upload quant_config.json
615aae3 verified
raw
history blame contribute delete
124 Bytes
{"load_in_4bit": true, "bnb_4bit_quant_type": "nf4", "bnb_4bit_compute_dtype": "float16", "bnb_4bit_use_double_quant": true}