---
license: mit
language:
- en
---

Submission to the BabyLM challenge 2024 trained on [Baby-cosmo-fine-100M](https://huggingface.co/datasets/ltg/babylm-2024-baby-cosmo-fine-100m).

The training scripts are published here: https://github.com/ltgoslo/gpt-bert

```latex
@misc{charpentier2024gptbertboth,
      title={GPT or BERT: why not both?}, 
      author={Lucas Georges Gabriel Charpentier and David Samuel},
      year={2024},
      eprint={2410.24159},
      archivePrefix={arXiv},
      primaryClass={cs.CL},
      url={https://arxiv.org/abs/2410.24159}, 
}
```