MusicLDM
Text-to-music latent diffusion that fights training-data plagiarism with beat-synchronous mixup augmentation.
About
MusicLDM is a text-to-music latent diffusion model that adapts the Stable Diffusion and AudioLDM playbook to the music domain, retraining both the CLAP text-audio encoder and the HiFi-GAN vocoder on music data. Its signature idea is about originality: because generative music models can memorize and regurgitate their training data, the authors propose beat-synchronous mixup augmentation, blending training samples in both the audio and latent domains on the beat, to push the model toward novel outputs that stay faithful to the requested style.
Trained on roughly ten thousand songs (466 hours of music), the model was published at ICASSP 2024 and ships inside Hugging Face diffusers, with the authors introducing new CLAP-score-based metrics to measure novelty and text-music correspondence. A practical caveat: the repository badges claim a CC BY-NC-SA 4.0 license, which would restrict use to non-commercial research, but this is only attested via badges rather than a verified license file, so confirm before any commercial plans.
Specs
Context window
Pricing
Free quota
API access
Weights
Released
Benchmarks
- Paper (arXiv 2308.01546): reports improved quality, novelty and text-music correspondence vs baselines on CLAP-score-based metrics (author-reported)
- ICASSP 2024 publication
- No independently verified headline numbers
Provider-reported unless a source says otherwise. Benchmarks are marketing-adjacent by nature — treat them as directional.
Use cases
Trust score — 6/10
Scored 6/10 (TIER metrics, metrics-v1). Evidence: ICASSP 2024 paper (arXiv 2308.01546) with CLAP-score-based novelty metrics; integrated into Hugging Face diffusers; checkpoints on Hugging Face (ucsd-reach/musicldm). Strengths: peer-reviewed, diffusers integration, novelty-focused design. Docked for: (1) license only attested via repo badge (CC BY-NC-SA 4.0) — not verified against an official license file, treat as non-commercial until confirmed; (2) headline claims are author-reported; (3) small community and an intermittently unavailable demo. License: CC BY-NC-SA 4.0 (badge-attested, unverified).
How the trust score is built
Full methodology →- Adoption · stars/downloads, log-scaled vs peers
- 6.0/10
- Maintenance · recency, releases, issue load
- 4.0/10
- Openness · license + docs
- 6.0/10
- Manual review · sanity read + red-flag check
- 6.0/10
- · ICASSP 2024 paper (arXiv 2308.01546): beat-synchronous mixup for novelty; trained on ~10k songs / 466h
- · Integrated into Hugging Face diffusers; checkpoints on HF (ucsd-reach/musicldm)
- · License only attested via repo badges — treat as non-commercial until confirmed; demo intermittently down