Quark installation doesn't work
#13
by
JacobAMD
- opened
In the page Model card,
I follow the instruction to download Quark-0.6.0.zip (https://quark.docs.amd.com/latest/install.html) and then install the wheel.
When I run the script:
python3 quantize_quark.py \
--model_dir /data/huggingface/hub/meta-llama/Llama-3.1-70B/
--output_dir Llama-3.1-70B-FP8 \
--quant_scheme w_fp8_a_fp8
--kv_cache_dtype fp8
--num_calib_data 128
--model_export quark_safetensors
--no_weight_matrix_merge
The error message shows that "from pytorchlight.config.configs_sets import int_kModuleNotFoundError: No module named 'pytorchlight' "