| datasets: | |
| - PatrickHaller/dsir-pile-100M-words | |
| language: | |
| - en | |
| library_name: transformers | |
| Our model for the 2024 BabyLM challenge 100M words track. | |
| To download and use this model the [fla](https://github.com/sustcsonglin/flash-linear-attention) package has to be installed: | |
| ```bash | |
| pip install -U git+https://github.com/sustcsonglin/flash-linear-attention | |
| ``` |