BitNet b1.58 2B4T

BitNet b1.58 2B4T is Microsoft Research's proof that large language models can be trained natively in 1 bit. Every weight is ternary, restricted to -1, 0 or 1, which means matrix multiplications become additions and the whole 2-billion-parameter model fits in about 0.4GB of memory excluding embeddings. It was trained from scratch on 4 trillion tokens, then instruction-tuned with SFT and DPO, and released in April 2025 under the MIT license with three weight formats: packed 1.58-bit for inference, BF16 masters for training, and GGUF for bitnet.cpp.

The technical report positions it as comparable to full-precision 2B-class open models, with reported scores of 53.17 on MMLU, 58.38 on GSM8K and 49.91 on ARC-Challenge, plus up to 82% lower energy use on x86 CPUs. It is the landmark result in the 1-bit line that started with the 2024 BitNet paper, and the tooling is genuinely usable: bitnet.cpp runs it on ordinary CPUs, and Hugging Face Transformers added native support. The context window is a modest 4K and it will not trade blows with modern 7B models, but as an efficiency demonstration it has no peer in the open.

• Microsoft technical report (arXiv:2504.12285), via secondary summaries: ARC-Challenge 49.91, GSM8K 58.38, MMLU 53.17, CommonsenseQA 71.58
• Microsoft-reported: up to 82.2% energy reduction on x86 CPUs vs full-precision models (provider claim)
• Positioned as comparable to full-precision 2B-class open models (provider claim)