MLC LLM
Deploy LLMs natively anywhere via machine-learning compilation on Apache TVM Unity.
About
MLC LLM is an open-source (Apache 2.0) universal LLM deployment engine built on Apache TVM Unity. Instead of shipping separate runtimes per platform, it compiles models with ML compilation — dynamic-shape IR, operator fusion, memory planning, and low-bit quantization — into native libraries that run on iOS, Android, WebGPU browsers, Windows, Linux, and macOS, including Apple Silicon, AMD, NVIDIA, and Intel GPUs.
It exposes an OpenAI-compatible API on every target, with a C++ CLI and reference chat apps showing how to embed compiled models. A companion project, WebLLM, brings the same pipeline to the browser. With ~22.8k GitHub stars, it is the reference implementation for the universal-native-deployment philosophy.
What it can do
- Compile any HF model into native libraries via TVM Unity ML compilation
- Run the same model on iOS, Android, WebGPU, and desktop GPUs
- OpenAI-compatible chat/completions API on every platform
- Low-bit quantization and dynamic-shape optimization built in
Setup
pip install mlc-llm or use the C++ CLI / iOS reference app; precompiled model weights available from the project.
Prerequisites
Target device (iOS, Android, desktop, or WebGPU browser); Python 3.9+ for the compile pipeline.
Use cases
Works with
Trust score — 8/10
Scored 8/10 (verified tier, metrics-v1). Evidence: Verified via vendor site/docs/GitHub and reputable coverage, wave 73; Trending score 84/100 from researcher assessment.
Sources
Every fact above was checked against these sources. Where a claim comes from the provider and couldn't be independently verified, the text says so.