Update README.md

README.md

@@ -16,6 +16,8 @@ Who needs em, we all have em, they're just like us. Unusable models, compute opt
 
 The B, C, and D classes are derived from the tokens per model ratio from LLaMA, as LLaMA 65B is nearly Chinchilla-optimal with a ratio of 21 x Million Params tokens in training. Descending down the model sizes per training set for each model gives us these classes.
 
+I also come up with a new pretraining method inspired by UL2, the only difference is it's 10:48pm so I don't have the patience to look at the implementation to see if every detail is correct. Calling it Blender in case it's too different.
+
 | Model Name | Parameters | Class | Ratio | Tokens | Batch Size (Tokens) | Training Loss | 
 | --- | --- | --- | --- | --- |  --- |  --- | 
 | GerbilLab/Gerbil-A-3.3m | 3.3m | A-Class | 20 | 60M | 65.5k | 6.6644 |
@@ -28,5 +30,6 @@ The B, C, and D classes are derived from the tokens per model ratio from LLaMA,
 | GerbilLab/Gerbil-A-15m | 15m | A-Class | 20 | 280M | 131k | 4.9999 |
 | GerbilLab/Gerbil-A-32m | 32m | A-Class | 20 | 640M | 262K | 4.0487 |
 | --- | --- | --- | --- | --- |  --- |  --- | 
+| GerbilLab/Gerbil-Blender-A-15m | 15m | A-Class | 20 | 280M | 131k | coming soon | 
 
 The only application where I can imagine these being useful in the slightest is warm-starting very small encoder-decoder models or fitting a new scaling law that takes into account smaller models. Every model was trained on a singular GPU, either a RTX2060, RTX3060, or a T4.