Achal Dave commited on
Commit
2fbae44
·
1 Parent(s): 62f139c
Files changed (1) hide show
  1. README.md +2 -2
README.md CHANGED
@@ -13,13 +13,13 @@ DCLM-1B is a 1.4 billion parameter language model trained on the DCLM-Baseline d
13
 
14
  ## Evaluation
15
 
16
- Here are the evaluation results for DCLM-1B on various tasks (using [llm-foundry](https://github.com/mosaicml/llm-foundry) eval suite), compared to recently released small models on key benchmarks.
17
  As described in the paper, Core accuracy is the average of centered accuracy on
18
  22 tasks (including HellaSwag and ARC-E), Extended is centered accuracy averaged
19
  over 53 tasks.
20
 
21
 
22
- | Model | Params | Tokens | Open dataset? | Core | MMLU | Extended |
23
  |-----------------------------------|--------|--------|---------------|----------|----------|-----------|
24
  | **Open weights, closed datasets** | | | | | | |
25
  | Qwen2-1.5B | 1.5B | ? | ❌ | 42.1 | **56.4** | **32.4** |
 
13
 
14
  ## Evaluation
15
 
16
+ We evaluate DCLM-1B using the [llm-foundry](https://github.com/mosaicml/llm-foundry) eval suite, and compare to recently released small models on key benchmarks.
17
  As described in the paper, Core accuracy is the average of centered accuracy on
18
  22 tasks (including HellaSwag and ARC-E), Extended is centered accuracy averaged
19
  over 53 tasks.
20
 
21
 
22
+ | Model | Params | Tokens | Open dataset? | Core | MMLU 5-shot | Extended |
23
  |-----------------------------------|--------|--------|---------------|----------|----------|-----------|
24
  | **Open weights, closed datasets** | | | | | | |
25
  | Qwen2-1.5B | 1.5B | ? | ❌ | 42.1 | **56.4** | **32.4** |