MeLoDy

MeLoDy, short for music plus language model plus diffusion, was ByteDance's SAMI team answer to MusicLM's biggest weakness: speed. It keeps MusicLM's top-level semantic language model for musical structure, then replaces the expensive acoustic stages with a novel dual-path diffusion model and an audio VAE-GAN. The result, per the May 2023 paper, is state-of-the-art quality with 95.7 to 99.6 percent fewer forward passes than MusicLM for 10 and 30-second clips, running faster than real time on a V100.

It was trained on 257,000 hours of music, takes text and audio prompts, supports audio continuation and can extend generations indefinitely. Notably, the evaluation included music producers, not just metrics. The catch is availability: no code or weights were ever released, so the efficiency claims rest on the paper alone.

Provider-reported: musicality, audio quality and text-relevance on par with MusicLM and Noise2Music, exceeding baselines on audio quality; producer evaluations.