neuralmagic
/

Meta-Llama-3.1-405B-Instruct-FP8

Text Generation

text-generation-inference

Inference Endpoints

compressed-tensors

Model card Files Files and versions Community

alexmarques commited on Jul 30, 2024

Commit

8318e06

·

verified ·

1 Parent(s): da5a62e

Update README.md

Files changed (1) hide show

README.md +2 -1

README.md CHANGED Viewed

@@ -285,6 +285,7 @@ lm_eval \
   --model_args pretrained="neuralmagic/Meta-Llama-3.1-405B-Instruct-FP8",dtype=auto,add_bos_token=True,max_model_len=4096,tensor_parallel_size=8 \
   --tasks gsm8k_cot_llama_3.1_instruct \
   --apply_chat_template \
   --num_fewshot 8 \
   --batch_size auto
 ```
@@ -314,7 +315,7 @@ lm_eval \
 lm_eval \
   --model vllm \
   --model_args pretrained="neuralmagic/Meta-Llama-3.1-405B-Instruct-FP8",dtype=auto,add_bos_token=True,max_model_len=4096,tensor_parallel_size=8 \
-  --tasks truthfulqa_mc \
   --num_fewshot 0 \
   --batch_size auto
 ```

   --model_args pretrained="neuralmagic/Meta-Llama-3.1-405B-Instruct-FP8",dtype=auto,add_bos_token=True,max_model_len=4096,tensor_parallel_size=8 \
   --tasks gsm8k_cot_llama_3.1_instruct \
   --apply_chat_template \
+  --fewshot_as_multiturn \
   --num_fewshot 8 \
   --batch_size auto
 ```
 lm_eval \
   --model vllm \
   --model_args pretrained="neuralmagic/Meta-Llama-3.1-405B-Instruct-FP8",dtype=auto,add_bos_token=True,max_model_len=4096,tensor_parallel_size=8 \
+  --tasks truthfulqa \
   --num_fewshot 0 \
   --batch_size auto
 ```