Moshi
First open-source real-time full-duplex spoken LLM — listens and speaks simultaneously, ~200ms latency.
About
Moshi by Kyutai Labs is the first open-source real-time full-duplex spoken dialogue model. It treats conversation as native speech-to-speech generation, eliminating the cascaded ASR → LLM → TTS pipeline and its compounding latency. The architecture combines Helium (7B text LLM), Mimi (streaming neural audio codec), and dual-stream modeling that processes user audio and generates output simultaneously — enabling interruptions, overlap, and backchanneling natively at ~160-200ms latency.
The honest picture: running Moshi needs a solid GPU (7B+ parameters), and the base model is English-focused with more limited multilingual ability than commercial alternatives. Full-duplex is genuinely harder to control than turn-based systems — barge-in behavior can feel unpredictable. But as a research artifact and foundation for real-time voice agents, it's unmatched in the open-source world.
What it can do
- Full-duplex speech-to-speech dialogue
- ~200ms practical latency with interruption handling
- Mimi streaming neural audio codec
- Inner monologue for improved factuality
- Streaming ASR and TTS modes from the same architecture
Setup
Clone kyutai-labs/moshi from GitHub and follow the README — needs PyTorch and a CUDA GPU. A live demo has historically been available on the Kyutai site.
Prerequisites
Python, PyTorch, CUDA-capable GPU with 12GB+ VRAM for the full model.
Use cases
Works with
Trust score — 8/10
Scored 8/10 (verified tier, metrics-v1). Evidence: Verified via vendor site/docs/GitHub and reputable coverage, wave 53; Trending score 80/100 from researcher assessment.
Sources
Every fact above was checked against these sources. Where a claim comes from the provider and couldn't be independently verified, the text says so.