OLMoE-1B-7B
Fully open 7B mixture-of-experts that runs at the cost of a 1B model.
About
OLMoE-1B-7B is AI2's sparse mixture-of-experts model: 7B total parameters with only about 1B active per token across 64 small experts. What sets it apart is total openness — weights, training data, code, and logs are all published, a rarity among MoE models.
AI2 reports it matching or beating larger models like Llama 2 13B on MMLU (54.1%) at a fraction of the inference cost. The 4K context window is modest, and as a 2024 model it is outclassed by newer small models, but it remains a reference point for open MoE research.
Specs
Context window
Pricing
Free quota
API access
Weights
Released
Benchmarks
MMLU 54.1% (AI2-reported)
Provider-reported unless a source says otherwise. Benchmarks are marketing-adjacent by nature — treat them as directional.
Use cases
Trust score — 8/10
Scored 8/10 (verified tier, metrics-v1). Evidence: Paper (arXiv 2409.02060) plus HF model card; full data/code/logs release; MMLU 54.1% is AI2-reported; Manual review: reference open-MoE work; modest 4K context, dated versus 2025 small models. License: Fully open (Apache 2.0 per AI2).
How the trust score is built
Full methodology →- Adoption · stars/downloads, log-scaled vs peers
- 7.0/10
- Maintenance · recency, releases, issue load
- 8.0/10
- Openness · license + docs
- 9.0/10
- Manual review · sanity read + red-flag check
- 8.0/10
- · Paper (arXiv 2409.02060) plus HF model card; full data/code/logs release
- · MMLU 54.1% is AI2-reported
- · Manual review: reference open-MoE work; modest 4K context, dated versus 2025 small models