update

dist/index.html

@@ -639,15 +639,17 @@
     </div>
     <p>Here are the key highlights of the ablation results above:</p>
     <ul>
-        <li>📚 FineWeb-Edu surpasses 🍷 FineWeb and all other open web datasets, with remarkable improvements on educational benchmarks such as MMLU, ARC, and OpenBookQA.</li>
+        <li>📚 FineWeb-Edu <strong>surpasses 🍷 FineWeb and all other open web datasets, with remarkable improvements on educational benchmarks</strong> such as MMLU, ARC, and OpenBookQA.</li>
         <li>It achieves the same performance with significantly less data, requiring 10x fewer tokens compared to C4 and Dolma to match MMLU results.</li>
         <li>This demonstrates the effectiveness of using classifiers trained on LLM annotations for large-scale data filtering.</li>
     </ul>
     <p>Given that a threshold of 2 also demonstrated strong performance while retaining more data, we are releasing an additional dataset filtered with this threshold, containing 5.4 trillion tokens under <a href="https://huggingface.co/datasets/HuggingFaceFW/fineweb-edu-score-2">HuggingFaceFW/fineweb-edu-score-2</a>.</p>
     <p>You can find the two datasets along with the classifier used for the filtering in this <a href="https://huggingface.co/collections/HuggingFaceFW/fineweb-edu-6659c3f3d399d0e1d648adfd">collection</a>.</p>
+
     <h2>Next steps</h2>
-    <p>Through our open data efforts we hope to give every model trainer the ability to create state-of-the-art large language models. As part of this process, we plan to continue iterating on FineWeb and to release more specialised filtered subsets of web data, in a fully open and reproducible manner.</p>
-    <p>While English currently dominates the large language model landscape, we believe that making high quality training data for other languages more easily accessible would allow millions of non english speakers to benefit from these technologies and, as such, will also strive to adapt the FineWeb Recipe to a multilingual version.</p>
+    <p>Through our open science efforts we hope to open more and more the block box aronud training good quality large language models as well as give every model trainer the ability to create state-of-the-art LLMs. We're excited to continue iterating on FineWeb and increasingly better filtered subsets of web data, in a fully open and reproducible manner.</p>
+    <p>Moreover, while English currently dominates the large language model landscape, we're looking forward to applying the learnings we make in project like this one to make high quality training data available as well in other languages and more easily accessible.</p>
+    <p>In a nutshell: the future is bright and exciting for studying the science of creating datasets at scale in the open.</p>
 </d-article>
 
 <d-appendix>

src/clusters.js

@@ -12,7 +12,7 @@ const DEFAULT_XAXIS = {
     showgrid: false,
     zeroline: false,
     title: {
-        text: "The 🍷 FineWeb dataset, <a href='https://github.com/huggingface/text-clustering' target='_blank' style='color: inherit;'>clustered</a> and annotated with educational score labels",
+        text: "<a href='https://github.com/huggingface/text-clustering' target='_blank' style='color: inherit;'>The 🍷 FineWeb dataset, clustered and annotated with educational score labels</a>",
         font: {
             size: 16,
             style: "italic",

src/index.html

@@ -639,15 +639,17 @@
     </div>
     <p>Here are the key highlights of the ablation results above:</p>
     <ul>
-        <li>📚 FineWeb-Edu surpasses 🍷 FineWeb and all other open web datasets, with remarkable improvements on educational benchmarks such as MMLU, ARC, and OpenBookQA.</li>
+        <li>📚 FineWeb-Edu <strong>surpasses 🍷 FineWeb and all other open web datasets, with remarkable improvements on educational benchmarks</strong> such as MMLU, ARC, and OpenBookQA.</li>
         <li>It achieves the same performance with significantly less data, requiring 10x fewer tokens compared to C4 and Dolma to match MMLU results.</li>
         <li>This demonstrates the effectiveness of using classifiers trained on LLM annotations for large-scale data filtering.</li>
     </ul>
     <p>Given that a threshold of 2 also demonstrated strong performance while retaining more data, we are releasing an additional dataset filtered with this threshold, containing 5.4 trillion tokens under <a href="https://huggingface.co/datasets/HuggingFaceFW/fineweb-edu-score-2">HuggingFaceFW/fineweb-edu-score-2</a>.</p>
     <p>You can find the two datasets along with the classifier used for the filtering in this <a href="https://huggingface.co/collections/HuggingFaceFW/fineweb-edu-6659c3f3d399d0e1d648adfd">collection</a>.</p>
+
     <h2>Next steps</h2>
-    <p>Through our open data efforts we hope to give every model trainer the ability to create state-of-the-art large language models. As part of this process, we plan to continue iterating on FineWeb and to release more specialised filtered subsets of web data, in a fully open and reproducible manner.</p>
-    <p>While English currently dominates the large language model landscape, we believe that making high quality training data for other languages more easily accessible would allow millions of non english speakers to benefit from these technologies and, as such, will also strive to adapt the FineWeb Recipe to a multilingual version.</p>
+    <p>Through our open science efforts we hope to open more and more the block box aronud training good quality large language models as well as give every model trainer the ability to create state-of-the-art LLMs. We're excited to continue iterating on FineWeb and increasingly better filtered subsets of web data, in a fully open and reproducible manner.</p>
+    <p>Moreover, while English currently dominates the large language model landscape, we're looking forward to applying the learnings we make in project like this one to make high quality training data available as well in other languages and more easily accessible.</p>
+    <p>In a nutshell: the future is bright and exciting for studying the science of creating datasets at scale in the open.</p>
 </d-article>
 
 <d-appendix>