ModernBERT
Modernized BERT encoder with 8K context for retrieval and classification
About
ModernBERT is the modernized bidirectional encoder from Answer.AI and collaborators, released December 2024 under Apache 2.0. Trained on 2 trillion tokens of English text and code, it stretches to 8,192 tokens of context and was the first base-size model to beat DeBERTaV3 on GLUE, per the team's report. It's an encoder, so it does classification, retrieval, and embeddings — not chat — and non-English performance lags.
Specs
Context window
Pricing
Free quota
API access
Weights
Released
Benchmarks
Team-reported: first base-size model to beat DeBERTaV3 on GLUE; state-of-the-art long-context retrieval with ColBERT.
Provider-reported unless a source says otherwise. Benchmarks are marketing-adjacent by nature — treat them as directional.
Use cases
Trust score — 8/10
Scored 8/10 (verified tier, metrics-v1). Evidence: Released 2024-12-18 by Answer.AI with LightOn, NVIDIA and others; Apache 2.0 confirmed on HF card; Encoder-only; 2T training tokens; 8,192-token context; GeGLU + RoPE + alternating attention; GLUE/DeBERTaV3 and retrieval claims are team-reported and labeled as such; entry notes it is not a chat model. License: Apache-2.0 confirmed on the Hugging Face card..
How the trust score is built
Full methodology →- Adoption · stars/downloads, log-scaled vs peers
- 8.0/10
- Maintenance · recency, releases, issue load
- 8.0/10
- Openness · license + docs
- 10.0/10
- Manual review · sanity read + red-flag check
- 7.0/10
- · Released 2024-12-18 by Answer.AI with LightOn, NVIDIA and others; Apache 2.0 confirmed on HF card
- · Encoder-only; 2T training tokens; 8,192-token context; GeGLU + RoPE + alternating attention
- · GLUE/DeBERTaV3 and retrieval claims are team-reported and labeled as such; entry notes it is not a chat model