A RoBERTa model trained using Data2Vec based on the paper data2vec: A General Framework for Self-supervised Learning in Speech, Vision and Language.
This model is provided here for this repo but was NOT trained using that codebase but instead, copied from facebook/data2vec-text-base for convenience and reproducibility.
BibTeX entry and citation info
@misc{https://doi.org/10.48550/arxiv.2202.03555,
doi = {10.48550/ARXIV.2202.03555},
url = {https://arxiv.org/abs/2202.03555},
author = {Baevski, Alexei and Hsu, Wei-Ning and Xu, Qiantong and Babu, Arun and Gu, Jiatao and Auli, Michael},
keywords = {Machine Learning (cs.LG), FOS: Computer and information sciences, FOS: Computer and information sciences},
title = {data2vec: A General Framework for Self-supervised Learning in Speech, Vision and Language},
publisher = {arXiv},
year = {2022},
copyright = {arXiv.org perpetual, non-exclusive license}
}