AudioLDM2

AudioLDM2 is an open-source text-to-audio generation model by Haohe Liu and collaborators, extending the original AudioLDM with a 'universal audio representation' approach. A single framework generates sound effects, music, and even speech from text prompts: the audioldm2-full checkpoint handles general audio and music, audioldm2-music-665k specializes in music, and dedicated speech checkpoints do text-to-speech from a transcription plus a speaker description. It is integrated with Hugging Face Diffusers and ships with a Gradio web app and CLI.

The honest picture: generation quality varies noticeably across hardware and random seeds — the authors themselves advise tuning the seed per machine — and outputs can sound synthetic next to newer commercial audio models. It is a research artifact from 2023 that newer open models have built upon. But as a fully open, locally runnable text-to-audio/music/speech pipeline with multiple specialized checkpoints, it remains a valuable foundation for experimentation and fine-tuning.