Trillion-7B

Trillion-7B is a Korean-targeted multilingual language model from Trillion Labs, a Korean startup founded in 2024. It introduced Cross-lingual Document Attention (XLDA), an architectural mechanism that transfers linguistic knowledge from a resource-rich language like English into Korean and other target languages without degrading English performance.

The headline claim is token efficiency: Trillion-7B reached competitive multilingual performance while dedicating only about 10 percent of its 2 trillion training tokens to multilingual data, with fewer than 180 billion Korean tokens. The full pretraining run reportedly cost just 59.4K H100 GPU hours, around $148K, making it one of the cheapest openly documented frontier-adjacent training runs.

Trillion Labs released Trillion-7B under Apache 2.0, and later continued the line with larger models including a 70B-class release with public intermediate checkpoints. The lab presented its Korean language model work at NVIDIA GTC 2025 and announced a June 2026 partnership with NVIDIA to develop industrial world models for AI data centers and power infrastructure, moving beyond language modeling.