Hugging Face
Models
Datasets
Spaces
Posts
Docs
Enterprise
Pricing
Log In
Sign Up
1
Sam
samheym
Follow
0 followers
ยท
3 following
AI & ML interests
None yet
Recent Activity
reacted
to
tomaarsen
's
post
with โค๏ธ
5 days ago
An assembly of 18 European companies, labs, and universities have banded together to launch ๐ช๐บ EuroBERT! It's a state-of-the-art multilingual encoder for 15 European languages, designed to be finetuned for retrieval, classification, etc. ๐ช๐บ 15 Languages: English, French, German, Spanish, Chinese, Italian, Russian, Polish, Portuguese, Japanese, Vietnamese, Dutch, Arabic, Turkish, Hindi 3๏ธโฃ 3 model sizes: 210M, 610M, and 2.1B parameters - very very useful sizes in my opinion โก๏ธ Sequence length of 8192 tokens! Nice to see these higher sequence lengths for encoders becoming more common. โ๏ธ Architecture based on Llama, but with bi-directional (non-causal) attention to turn it into an encoder. Flash Attention 2 is supported. ๐ฅ A new Pareto frontier (stronger *and* smaller) for multilingual encoder models ๐ Evaluated against mDeBERTa, mGTE, XLM-RoBERTa for Retrieval, Classification, and Regression (after finetuning for each task separately): EuroBERT punches way above its weight. ๐ Detailed paper with all details, incl. data: FineWeb for English and CulturaX for multilingual data, The Stack v2 and Proof-Pile-2 for code. Check out the release blogpost here: https://huggingface.co/blog/EuroBERT/release * https://huggingface.co/EuroBERT/EuroBERT-210m * https://huggingface.co/EuroBERT/EuroBERT-610m * https://huggingface.co/EuroBERT/EuroBERT-2.1B The next step is for researchers to build upon the 3 EuroBERT base models and publish strong retrieval, zero-shot classification, etc. models for all to use. I'm very much looking forward to it!
published
a model
12 days ago
samheym/GerBi-Encoder
updated
a model
12 days ago
samheym/GerBi-Encoder
View all activity
Organizations
None yet
models
3
Sort:ย Recently updated
samheym/GerBi-Encoder
Sentence Similarity
โข
Updated
12 days ago
โข
14
samheym/GerCross-Encoder
Text Classification
โข
Updated
12 days ago
โข
5
samheym/GerColBERT
Sentence Similarity
โข
Updated
21 days ago
โข
65
datasets
3
Sort:ย Recently updated
samheym/ger-dpr-collection-distillation
Viewer
โข
Updated
30 days ago
โข
600
โข
12
samheym/ger-dpr-collection
Viewer
โข
Updated
Jan 9
โข
40.5M
โข
27
samheym/ger-dpr-collection-crossencoder
Viewer
โข
Updated
Dec 19, 2024
โข
81.1M
โข
29