Nicolas-BZRD

Hey @CorentinAmbroise , we are currently working on the modeling file to add the different tasks required to execute the MTEB benchmark. We hope to achieve it soon.

We are working on the next model, which covers all European languages. Training the previous model with a restricted number of languages helped us better understand the impact of their distribution during training and the curse of multilinguality while maximizing population coverage.

We also released the code base and look forward to see the community adding more languages 🤗

ModernBERT is English-only. We achieve similar performance in English with our small model (which is slightly larger than ModernBERT) and better performance with our medium and large models. For multilingual tasks, we obtain superior results. However, since comparing ModernBERT on multilingual data is less meaningful, we chose not to report those results. For math and code, the comparison is more relevant, so we included it. However, you are right—we will add the results in the appendix.