jerryzh168 commited on
Commit
31f1a7e
·
verified ·
1 Parent(s): f6dfa8c

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +5 -7
README.md CHANGED
@@ -21,12 +21,15 @@ pipeline_tag: text-generation
21
 
22
  # Installation
23
  ```
24
- pip install transformers
25
  pip install --pre torchao --index-url https://download.pytorch.org/whl/nightly/cu126
26
- pip install [email protected]:EleutherAI/lm-evaluation-harness.git
27
  pip install vllm --pre --extra-index-url https://wheels.vllm.ai/nightly
28
  ```
29
 
 
 
 
 
30
  # Quantization Recipe
31
  We used following code to get the quantized model:
32
 
@@ -81,11 +84,6 @@ print(f"{save_to} model:", benchmark_fn(quantized_model.generate, **inputs, max_
81
  # Model Quality
82
  We rely on [lm-evaluation-harness](https://github.com/EleutherAI/lm-evaluation-harness) to evaluate the quality of the quantized model.
83
 
84
- ## Installing the nightly version to get most recent updates
85
- ```
86
- pip install git+https://github.com/EleutherAI/lm-evaluation-harness
87
- ```
88
-
89
  ## baseline
90
  ```
91
  lm_eval --model hf --model_args pretrained=microsoft/Phi-4-mini-instruct --tasks hellaswag --device cuda:0 --batch_size 8
 
21
 
22
  # Installation
23
  ```
24
+ pip install git+https://github.com/huggingface/transformers
25
  pip install --pre torchao --index-url https://download.pytorch.org/whl/nightly/cu126
 
26
  pip install vllm --pre --extra-index-url https://wheels.vllm.ai/nightly
27
  ```
28
 
29
+ Also need to install lm-eval from source:
30
+ https://github.com/EleutherAI/lm-evaluation-harness#install
31
+
32
+
33
  # Quantization Recipe
34
  We used following code to get the quantized model:
35
 
 
84
  # Model Quality
85
  We rely on [lm-evaluation-harness](https://github.com/EleutherAI/lm-evaluation-harness) to evaluate the quality of the quantized model.
86
 
 
 
 
 
 
87
  ## baseline
88
  ```
89
  lm_eval --model hf --model_args pretrained=microsoft/Phi-4-mini-instruct --tasks hellaswag --device cuda:0 --batch_size 8