MusicLM

MusicLM is Google's text-to-music generation model, introduced in a January 2023 paper. It casts music generation as a hierarchical sequence-to-sequence task: MuLan embeddings condition the prompt, a transformer generates semantic tokens capturing melody and structure, and a second stage produces fine acoustic tokens decoded to 24 kHz audio that stays coherent over several minutes. It could also transform whistled or hummed melodies into a described style.

In evaluations it outperformed previous text-to-music systems on both audio quality and adherence to the description, and Google released the MusicCaps dataset — 5,500 expert-written music-text pairs — to support further research.

An honest caveat: Google never publicly released MusicLM itself, citing concerns about training-data rights and memorization. It remains a research artifact rather than a usable product — the ideas live on in later systems, but you cannot download or query the original model.