ZipVoice

ZipVoice is a zero-shot text-to-speech model from Xiaomi that attacks the size and speed problem head on. Built on conditional flow matching, it replaces the usual large transformer or diffusion backbone with Zipformer, an architecture originally designed for speech recognition, and aligns text to speech with a deliberately simple trick: assume every text token lasts about the same time and upsample uniformly, skipping the duration model entirely. A distilled variant, ZipVoice-Distill, cuts sampling steps and removes classifier-free-guidance overhead for even faster inference.

The paper reports that despite being three times smaller and up to thirty times faster than a DiT-based flow-matching baseline, ZipVoice matches state-of-the-art zero-shot TTS on intelligibility, speaker similarity and naturalness, trained on 100k hours of multilingual data. It handles English and Chinese with a compact 123M checkpoint, and the code plus checkpoints are Apache-2.0. A dialogue-oriented follow-up, ZipVoice-Dialog, extends the same ideas to spoken conversation.

• Paper (arXiv 2506.13053): 3x smaller and up to 30x faster than a DiT-based flow-matching baseline at comparable intelligibility, speaker similarity and naturalness
• Trained on 100k hours of multilingual data
• Author-reported; no independent verification