|
--- |
|
license: mit |
|
language: |
|
- en |
|
--- |
|
|
|
Submission to the BabyLM challenge 2024 trained on [Baby-cosmo-fine-100M](https://huggingface.co/datasets/ltg/babylm-2024-baby-cosmo-fine-100m). |
|
|
|
The training scripts are published here: https://github.com/ltgoslo/gpt-bert |
|
|
|
```latex |
|
@misc{charpentier2024gptbertboth, |
|
title={GPT or BERT: why not both?}, |
|
author={Lucas Georges Gabriel Charpentier and David Samuel}, |
|
year={2024}, |
|
eprint={2410.24159}, |
|
archivePrefix={arXiv}, |
|
primaryClass={cs.CL}, |
|
url={https://arxiv.org/abs/2410.24159}, |
|
} |
|
``` |
