Make-An-Audio
Prompt-enhanced diffusion for text-to-audio, extended to any-modality inputs.
About
Make-An-Audio (ICML 2023) tackled two bottlenecks in text-to-audio: scarce text-audio pairs and the difficulty of modeling long continuous audio. Its prompt-enhanced diffusion model uses a distill-then-reprogram trick to multiply concept combinations from language-free audio, plus a spectrogram autoencoder and CLAP representations. The paper claimed state of the art on objective and subjective benchmarks and introduced X-to-Audio generation from user-defined modalities, billed as no modality left behind.
Make-An-Audio 2 (May 2023, with ByteDance interns among the authors) focused on time: LLMs parse prompts into structured event-and-order pairs, a feed-forward Transformer denoiser handles variable-length generation, and LLM-augmented data eases temporal data scarcity. Both papers report strong gains on standard benchmarks, but no official public code or weights release could be found, so independent verification is thin.
Specs
Context window
Pricing
Free quota
API access
Weights
Released
Benchmarks
SOTA claims on AudioCaps/AudioSet objective and subjective evals (provider-reported, v1); temporal-understanding gains (provider-reported, v2).
Provider-reported unless a source says otherwise. Benchmarks are marketing-adjacent by nature — treat them as directional.
Use cases
Trust score — 6/10
Scored 6/10 (verified tier, metrics-v1). Evidence: ICML 2023 paper plus a temporal-enhanced v2, strong reported benchmark results. Strengths: clever data-scarcity workaround, any-modality input framing, v2 temporal modeling. Docked for: (1) no official public code or weights release found; (2) all benchmarks provider-reported. License: research papers; no public release located.
How the trust score is built
Full methodology →- Adoption · stars/downloads, log-scaled vs peers
- 6.0/10
- Maintenance · recency, releases, issue load
- 5.0/10
- Openness · license + docs
- 2.0/10
- Manual review · sanity read + red-flag check
- 6.0/10
- · ICML 2023; prompt-enhanced diffusion with CLAP (arXiv 2301.12661)
- · Make-An-Audio 2 (May 2023): LLM-parsed event-order pairs, temporal modeling
- · Variable-length, X-to-Audio generation claims
- · No official public code or weights release found
- · No logo staged (research project, no brand mark)