Update README.md

README.md

@@ -3,6 +3,7 @@ license: apache-2.0
 datasets:
 - allenai/dolma
 - allenai/tulu-v2-sft-mixture
+- allenai/ultrafeedback_binarized_cleaned
 language:
 - en
 ---
@@ -11,7 +12,7 @@ language:
 <img src="https://allenai.org/olmo/olmo-7b-animation.gif" alt="OLMo Logo" width="800" style="margin-left:'auto' margin-right:'auto' display:'block'"/>
 
 
-# Model Card for OLMo 1.7 7B SFT
+# Model Card for OLMo 7B April 2024 SFT
 
 **Requires transformers versions v4.40.0 or newer**
 
@@ -20,7 +21,8 @@ OLMo is a series of **O**pen **L**anguage **Mo**dels designed to enable the scie
 The OLMo base models are trained on the [Dolma](https://huggingface.co/datasets/allenai/dolma) dataset.
 The adapted versions are trained on the [Tulu SFT mixture](https://huggingface.co/datasets/allenai/tulu-v2-sft-mixture) and, for the Instruct version, a [cleaned version of the UltraFeedback dataset](https://huggingface.co/datasets/allenai/ultrafeedback_binarized_cleaned).
 
-OLMo 1.7 7B Instruct and OLMo SFT are two adapted versions of these models trained for better question answering.
+OLMo 7B April 2024 Instruct and OLMo SFT are two adapted versions of these models trained for better question answering.
+They are based on the OLMo 7B April release (previously called OLMo 1.7).
 They show the performance gain that OLMo base models can achieve with existing fine-tuning techniques.
 
 ## Model Details
@@ -28,13 +30,13 @@ They show the performance gain that OLMo base models can achieve with existing f
 We release two adapted model versions:
 | Model | Training Method(s) | Datasets | Context Length | 
 |------|--------|---------|--|
-| [OLMo 1.7 7B SFT](https://huggingface.co/allenai/OLMo-1.7-7B-SFT-hf)   | SFT | [Tulu 2 SFT Mix](https://huggingface.co/datasets/allenai/tulu-v2-sft-mixture)     | 2048        | 
-| [OLMo 1.7 7B Instruct](https://huggingface.co/allenai/OLMo-1.7-7B-Instruct-hf) | SFT + DPO   | [Tulu 2 SFT Mix](https://huggingface.co/datasets/allenai/tulu-v2-sft-mixture) + [Ultrafeedback Cleaned](https://huggingface.co/datasets/allenai/ultrafeedback_binarized_cleaned)     | 2048 |
+| [OLMo 7B April 2024 SFT](https://huggingface.co/allenai/OLMo-1.7-7B-SFT-hf)   | SFT | [Tulu 2 SFT Mix](https://huggingface.co/datasets/allenai/tulu-v2-sft-mixture)     | 2048        | 
+| [OLMo 7B April 2024 Instruct](https://huggingface.co/allenai/OLMo-1.7-7B-Instruct-hf) | SFT + DPO   | [Tulu 2 SFT Mix](https://huggingface.co/datasets/allenai/tulu-v2-sft-mixture) + [Ultrafeedback Cleaned](https://huggingface.co/datasets/allenai/ultrafeedback_binarized_cleaned)     | 2048 |
 
-These models are both trained on top of OLMo 1.7 7b:
+These models are both trained on top of OLMo 7B April 2024 release (formerly called OLMo 1.7):
 | Size | Training Tokens | Layers | Hidden Size | Attention Heads | Context Length |
 |------|--------|---------|-------------|-----------------|----------------|
-| [OLMo 1.7 7B](https://huggingface.co/allenai/OLMo-1.7-7B-hf)   | 2.05 Trillion |32     | 4096        | 32              | 4096  |
+| [OLMo 7B April 2024](https://huggingface.co/allenai/OLMo-1.7-7B-hf)   | 2.05 Trillion |32     | 4096        | 32              | 4096  |
 
 
 ### Model Description
@@ -68,8 +70,8 @@ You can run these models using recent (>= 4.40) versions of transformers.
 
 ```python
 from transformers import AutoModelForCausalLM, AutoTokenizer
-olmo = AutoModelForCausalLM.from_pretrained("allenai/OLMo-1.7-7B-SFT-hf")
-tokenizer = AutoTokenizer.from_pretrained("allenai/OLMo-1.7-7B-SFT-hf")
+olmo = AutoModelForCausalLM.from_pretrained("allenai/OLMo-7B-0424-Instruct-hf")
+tokenizer = AutoTokenizer.from_pretrained("allenai/OLMo-7B-0424-Instruct-hf")
 chat = [
     { "role": "user", "content": "What is language modeling?" },
 ]
@@ -95,9 +97,10 @@ Core model results for the 7B adapted models are found below.
 
 | Model                 | MMLU 0-shot ↑ | AlpacaEval %win ↑ | ToxiGen % Toxic ↓ | TruthfulQA %Info+True ↑ |
 |-----------------------|---------------|--------------------|--------------------|-------------------------|
-| **OLMo 1.7 base**      | 47.5          | -                  | 83.2               | 25.7                    |
-| **[OLMo 1.7 7B SFT](https://huggingface.co/allenai/OLMo-1.7-7B-SFT-hf)**         |    52.4       | 70.4               | 0.5               | 38.8                    |
-| **[OLMo 1.7 7B Instruct](https://huggingface.co/allenai/OLMo-1.7-7B-Instruct-hf)**     | 52.4          | 82.2               | 0.2                | 75.6   
+| **OLMo 7B April 2024 base**      | 47.5          | -                  | 83.2               | 25.7                    |
+| **[OLMo 7B April 2024 SFT](https://huggingface.co/allenai/OLMo-1.7-7B-SFT-hf)**         |    52.4       | 70.4               | 0.5               | 38.8                    |
+| **[OLMo 7B April 2024 Instruct](https://huggingface.co/allenai/OLMo-1.7-7B-Instruct-hf)**     | 52.4          | 82.2               | 0.2                | 75.6                    |
+
 
 
 ## Model Details