CosyVoice 2

CosyVoice 2 is the FunAudioLLM project's streaming text-to-speech model from Alibaba's Tongyi Speech Lab. The 0.5B-parameter model combines a text/speech language model with chunk-aware flow matching to support streaming text input and audio output — you don't wait for the full sentence before audio starts. It does zero-shot voice cloning, multilingual synthesis, and instruction-following prosody control. The full stack (inference, training, deployment, FastAPI server) is open-source on GitHub, with vLLM and TensorRT-LLM runtime support contributed by the community.

The honest picture: setup is genuinely involved — Conda environment, submodules, and some dependency wrangling, with much of the documentation and community discussion in Chinese. On English benchmarks it trails closed APIs on naturalness, and 0.5B is small enough that complex prosody can sound flat. Note CosyVoice 3 (Fun-CosyVoice3) has since been released with better quality — check whether v2 or v3 fits your latency needs.

Clone github.com/FunAudioLLM/CosyVoice recursively, create the Conda env (Python 3.10), install requirements, download the CosyVoice2-0.5B checkpoint from Hugging Face or ModelScope.