Create README.md

README.md

@@ -0,0 +1,42 @@
+---
+language:
+- en
+---
+
+## EconBERTa: Towards Robust Extraction of Named Entities in Economics
+
+[EconBERTa](https://aclanthology.org/2023.findings-emnlp.774)
+EconBERTa is a DeBERTa-based language model trained from scratch in the domain of economics. It has been pretrained following the [ELECTRA](https://arxiv.org/abs/2003.10555) approach, using a large corpus consisting of 9,4B tokens from 1,5M economics papers (around 800,000 full articles and 700,000 abstracts).
+
+
+### Citation
+
+If you find EconBERTa useful for your work, please cite the following paper:
+
+``` latex
+@inproceedings{lasri-etal-2023-econberta,
+    title = "{E}con{BERT}a: Towards Robust Extraction of Named Entities in Economics",
+    author = "Lasri, Karim  and
+      de Castro, Pedro Vitor Quinta  and
+      Schirmer, Mona  and
+      San Martin, Luis Eduardo  and
+      Wang, Linxi  and
+      Dulka, Tom{\'a}{\v{s}}  and
+      Naushan, Haaya  and
+      Pougu{\'e}-Biyong, John  and
+      Legovini, Arianna  and
+      Fraiberger, Samuel",
+    editor = "Bouamor, Houda  and
+      Pino, Juan  and
+      Bali, Kalika",
+    booktitle = "Findings of the Association for Computational Linguistics: EMNLP 2023",
+    month = dec,
+    year = "2023",
+    address = "Singapore",
+    publisher = "Association for Computational Linguistics",
+    url = "https://aclanthology.org/2023.findings-emnlp.774",
+    doi = "10.18653/v1/2023.findings-emnlp.774",
+    pages = "11557--11577",
+    abstract = "Adapting general-purpose language models has proven to be effective in tackling downstream tasks within specific domains. In this paper, we address the task of extracting entities from the economics literature on impact evaluation. To this end, we release EconBERTa, a large language model pretrained on scientific publications in economics, and ECON-IE, a new expert-annotated dataset of economics abstracts for Named Entity Recognition (NER). We find that EconBERTa reaches state-of-the-art performance on our downstream NER task. Additionally, we extensively analyze the model{'}s generalization capacities, finding that most errors correspond to detecting only a subspan of an entity or failure to extrapolate to longer sequences. This limitation is primarily due to an inability to detect part-of-speech sequences unseen during training, and this effect diminishes when the number of unique instances in the training set increases. Examining the generalization abilities of domain-specific language models paves the way towards improving the robustness of NER models for causal knowledge extraction.",
+}
+```