OLMo 2 13B Instruct

OLMo 2 is the Allen Institute for AI's second-generation Open Language Model — and one of the most radically transparent models ever released: weights, training data (Dolma), training code, and evaluation code are all public under permissive licenses. The 13B instruct variant (RLVR2-tuned) is the one to use for chat and instruction-following.

Benchmarks (provider-reported — AllenAI's own eval tables on the HF model cards):

• Instruct variant (avg 62.0): GSM8k 87.4, IFEval 82.6, MMLU 68.5, AlpacaEval 39.5, MATH 39.2, BBH 58.8, DROP 71.5, Safety 79.1, TruthQA 64.3
• Same table: Llama-3.1-8B-Instruct averages 58.9 (GSM8k 83.4, MMLU 71.3, IFEval 80.6) — OLMo-2-13B-Instruct beats it on average, GSM8k, IFEval, and AlpacaEval, but trails on MMLU
• Base 13B (avg 68.3): ARC-C 83.5, HellaSwag 86.4, MMLU 67.5, GSM8K 75.1, DROP 70.7, AGIEval 54.2 — ahead of Llama-3.1-8B (avg 61.8) and Mistral-Nemo-12B (avg 66.9) on AllenAI's average, in the same harness
• No independent third-party re-verification found; all numbers are from AllenAI's harness

Practical notes: the 4K context window (verified in config.json) is short by current standards. The card itself warns of limited safety training — don't deploy it facing users without guardrails. Its real value is as a research artifact: anyone can reproduce, audit, and build on the full pipeline, which is why it's heavily cited in open-science work. Note: OLMo 3 (Oct 2026) now supersedes it.

Provider-reported (AllenAI HF model cards, their own harness):

Instruct 13B (avg 62.0): GSM8k 87.4 | IFEval 82.6 | MMLU 68.5 | AlpacaEval 39.5 | MATH 39.2 | BBH 58.8 | DROP 71.5 | Safety 79.1 | TruthQA 64.3

Vs Llama-3.1-8B-Instruct: avg 58.9, GSM8k 83.4, MMLU 71.3, IFEval 80.6

Base 13B (avg 68.3): ARC-C 83.5 | HSwag 86.4 | MMLU 67.5 | GSM8K 75.1 | DROP 70.7 | AGIEval 54.2

No independent benchmarks found.