Update README.md

README.md

@@ -75,9 +75,7 @@ In order to test our hypothesis, we first calculated the perplexity of each docu
 With the extracted perplexity percentiles, we created two functions to oversample the central quartiles with the idea of biasing against samples that are either too small (short, repetitive texts) or too long (potentially poor quality) (see Figure 3).
 
 The first function is a `Stepwise` that simply oversamples the central quartiles using quartile boundaries and a factor for the desired sampling frequency for each quartile, obviously given larger frequencies for middle quartiles (oversampling Q2, Q3, subsampling Q1, Q4). 
-The second function weighted the perplexity distribution by a Gaussian-like
-function, to smooth out the sharp boundaries of the `Stepwise` function and
-give a better approximation to the desired underlying distribution (see Figure 4).
+The second function weighted the perplexity distribution by a Gaussian-like function, to smooth out the sharp boundaries of the `Stepwise` function and give a better approximation to the desired underlying distribution (see Figure 4).
 
 We adjusted the `factor` parameter of the `Stepwise` function, and the `factor` and `width` parameter of the `Gaussian` function to roughly be able to sample 50M samples from the 416M in `mc4-es` (see Figure 4). For comparison, we also sampled randomly `mC4-es` up to 50M samples as well. In terms of sizes, we went down from 1TB of data to ~200GB.
 
@@ -86,18 +84,18 @@ We adjusted the `factor` parameter of the `Stepwise` function, and the `factor`
 
 ![](./images/perp-resample.png)
 
-<caption>Figure 3. Expected perplexity distributions of the sample `mc4-es` after applying the `Stepwise` function.</caption>
+<caption>Figure 3. Expected perplexity distributions of the sample mc4-es after applying the Stepwise function.</caption>
 </figure>
 
 <figure>
 
 ![](./images/perp-resample-gaussian.png)
 
-<caption>Figure 4. Expected perplexity distributions of the sample `mc4-es` after applying `Gaussian` function.</caption>
+<caption>Figure 4. Expected perplexity distributions of the sample mc4-es after applying Gaussian function.</caption>
 </figure>
 
 Figure 5 shows the actual perplexity distributions of the generated 50M subsets for
-each of the executed subsampling procedures. All subsets can be easily accessed for reproducibility purposes using the `bertin-project/mc4-es-sampled` dataset. Since the validation set was too small to extract a 10% (5M) of the samples using perplexity-sampling with the same `factor` and `width`, in our experiments we decided to sample from the training sets. In the `bertin-project/mc4-es-sampled` dataset, the `validation` set pulls the samples from the original `mc4`.
+each of the executed subsampling procedures. All subsets can be easily accessed for reproducibility purposes using the bertin-project/mc4-es-sampled dataset. Since the validation set was too small to extract a 10% (5M) of the samples using perplexity-sampling with the same "factor" and width, in our experiments we decided to sample from the training sets. In the bertin-project/mc4-es-sampled dataset, the validation set pulls the samples from the original mc4.
 
 ```python
 from datasets import load_dataset
@@ -118,9 +116,7 @@ for config in ("random", "stepwise", "gaussian"):
 
 ![](./images/datasets-perp.png)
 
-<caption>Figure 5. Experimental perplexity distributions of the sampled
-`mc4-es` after applying `Gaussian` and `Stepwise` functions, and the `Random`
-control sample.</caption>
+<caption>Figure 5. Experimental perplexity distributions of the sampled mc4-es after applying Gaussian and Stepwise functions, and the Random control sample.</caption>
 </figure>
 
 `Random` sampling displayed the same perplexity distribution of the underlying true distribution, as can be seen in Figure 6.
@@ -129,7 +125,7 @@ control sample.</caption>
 
 ![](./images/datasets-random-comparison.png)
 
-<caption>Figure 6. Experimental perplexity distribution of the sampled `mc4-es` after applying `Random` sampling.</caption>
+<caption>Figure 6. Experimental perplexity distribution of the sampled mc4-es after applying Random sampling.</caption>
 </figure>