hf (dtype=bfloat16,use_cache=True,pretrained=./checkpoint-1400/,max_length=2048), gen_kwargs: (None), limit: None, num_fewshot: None, batch_size: 16
|                Tasks                |Version|Filter|n-shot| Metric |   |Value |   |Stderr|
|-------------------------------------|-------|------|-----:|--------|---|-----:|---|-----:|
|leaderboard_gpqa                     |    N/A|      |      |        |   |      |   |      |
| - leaderboard_gpqa_diamond          |      1|none  |     0|acc_norm|↑  |0.3030|±  |0.0327|
| - leaderboard_gpqa_extended         |      1|none  |     0|acc_norm|↑  |0.3004|±  |0.0196|
| - leaderboard_gpqa_main             |      1|none  |     0|acc_norm|↑  |0.2969|±  |0.0216|
|leaderboard_musr                     |    N/A|      |      |        |   |      |   |      |
| - leaderboard_musr_murder_mysteries |      1|none  |     0|acc_norm|↑  |0.5400|±  |0.0316|
| - leaderboard_musr_object_placements|      1|none  |     0|acc_norm|↑  |0.3203|±  |0.0292|
| - leaderboard_musr_team_allocation  |      1|none  |     0|acc_norm|↑  |0.4080|±  |0.0311|

hf (dtype=bfloat16,use_cache=True,pretrained=./checkpoint-1400/,max_length=768), gen_kwargs: (None), limit: None, num_fewshot: None, batch_size: 128
|Tasks|Version|     Filter     |n-shot|  Metric   |   |Value |   |Stderr|
|-----|------:|----------------|-----:|-----------|---|-----:|---|-----:|
|gsm8k|      3|flexible-extract|     5|exact_match|↑  |0.5974|±  |0.0135|
|     |       |strict-match    |     5|exact_match|↑  |0.5921|±  |0.0135|