WebLLM
High-performance LLM inference engine that runs entirely inside the browser.
About
WebLLM is the web runtime for MLCEngine from the MLC team. It brings large language models into the browser with WebGPU hardware acceleration, so developers can build AI-powered web apps with no server-side processing and no data leaving the user's machine.
It exposes an OpenAI-compatible API, supports models like Llama, Phi, Gemma, and Mistral compiled by MLC-LLM, and runs inference inside web workers to keep the UI responsive. Because the same compiled core also powers native iOS and Android bindings, weights built once can serve across platforms.
What it can do
- In-browser LLM inference via WebGPU
- OpenAI-compatible chat completions API
- Streaming, JSON mode, and tool calling
- Web worker support for non-blocking inference
Setup
Install the npm package @mlc-ai/web-llm and create an MLCEngine with a prebuilt model ID, or try the hosted WebLLM Chat demo first.
Prerequisites
A WebGPU-capable browser (recent Chrome recommended); the model weights download on first use.
Use cases
Works with
Trust score — 8/10
Scored 8/10 (verified tier, metrics-v1). Evidence: Verified via vendor site/docs/GitHub and reputable coverage, wave 116; Trending score 84/100 from researcher assessment.
Sources
Every fact above was checked against these sources. Where a claim comes from the provider and couldn't be independently verified, the text says so.