EuroBERT

EuroBERT is a family of multilingual encoder models built by a European research collaboration led from CentraleSupélec's MICS laboratory with Diabolocom, Artefact, and Unbabel, supported by AMD and CINES. Introduced on March 10, 2025, it adapts Llama 3-style architecture (grouped-query attention, SwiGLU, RoPE) to bidirectional encoding.

The family ships in three sizes - 210M, 610M, and 2.1B parameters - trained on 5 trillion multilingual tokens covering 15 European and global languages plus code and math, with an 8,192-token native context. On multilingual retrieval and classification benchmarks the team reports it outperforming XLM-RoBERTa and mGTE variants.

Honest limits: encoders are embedding tools, not chat models - they do not generate text. Benchmark claims are from the authors' paper (arXiv 2503.05500). The family matters most for RAG, search, and classification over European languages.