Granite 3.0 8B Instruct

Granite 3.0 8B Instruct is IBM's 8-billion-parameter dense language model tuned for enterprise work — RAG pipelines, classification, summarization, code assistance, and function calling for agentic workflows. It was trained on a claimed 12 trillion tokens across 12 natural languages and 116 programming languages, with alignment done via IBM's InstructLab methodology (SFT + RL + model merging), and ships with a 128K context window under Apache 2.0.

Benchmarks (provider-reported — IBM Granite 3.0 paper, Table 10, same values on the HF model card):

• IFEval 52.27 (0-shot), MT-Bench 8.22, MMLU 65.82 (5-shot), MMLU-Pro 34.45
• BBH 51.55 (3-shot), GSM8K 68.99 (5-shot CoT), MATH 30.94
• HumanEval-Synthesis 64.63, HumanEval-Explain 57.16, HumanEval-Fix 65.85, MBPP 49.60
• ARC-C 64.16, GPQA 33.81, TruthfulQA 60.32
• In the same harness, Llama-3.1-8B scored MMLU 68.27, IFEval 50.37, GSM8K 65.04, MATH 34.46, HumanEval-Synthesis 63.41, BBH 50.81 — so Granite 3.0 8B trails on MMLU/MATH but beats Llama 3.1 8B on instruction-following, math word problems, and coding tasks. Cybersecurity benchmarks (IBM's specialty) are also claimed wins, per IBM's launch materials.
• No independent re-verification of these numbers was found; IBM also publishes Granite Guardian 3.0 safety models alongside.

Practical note: this is now two generations old — Granite 3.3 8B Instruct (Apr 2025) scores substantially higher across the board (e.g. HumanEval ~89.7, Arena-Hard 57.6 on IBM's HF card). The 3.0 release matters mainly as the widely-deployed baseline and the watsonx default.

Provider-reported (IBM Granite 3.0 paper Table 10; identical values on HF model card):

• IFEval 52.27 (0-shot) | MT-Bench 8.22 | MMLU 65.82 (5-shot) | MMLU-Pro 34.45
• BBH 51.55 (3-shot) | GSM8K 68.99 (5-shot CoT) | MATH 30.94
• HumanEval-Synthesis 64.63 / Explain 57.16 / Fix 65.85 | MBPP 49.60
• ARC-C 64.16 | GPQA 33.81 | TruthfulQA 60.32

Vs Llama-3.1-8B in same harness: MMLU 68.27, IFEval 50.37, GSM8K 65.04, MATH 34.46, HumanEval-Syn 63.41, BBH 50.81.

No independent benchmarks found.