Spirit LM

Spirit LM, introduced by Meta FAIR in early 2024, extends a 7-billion-parameter text model into the speech modality by continuously training on interleaved text and speech units. Speech and text are concatenated as a single token stream using a word-level interleaving method over a small curated speech-text parallel corpus.

It ships in Base and Expressive variants, the latter modeling pitch and style units alongside phonetic HuBERT units, and demonstrates few-shot learning across modalities including ASR, TTS, and speech classification. Meta released the model weights and inference code openly; verify the model card for the exact license before commercial use.