Skip to content
← Models
Open weightsAuffusion (open-source research project)/10 trustVerified

Auffusion

Text-to-audio latent diffusion built on Stable Diffusion, with open weights.

Visit official page →

About

Auffusion takes a pragmatic shortcut in text-to-audio: instead of training from scratch, it adapts text-to-image diffusion frameworks to audio, borrowing their generative strength and cross-modal alignment. The January 2024 paper shows this transfer works, with the model beating prior text-to-audio approaches on objective and subjective tests while using limited data and compute. It generates realistic human, animal, natural and artificial sounds plus sound effects from text prompts, and ships Colab notebooks for audio manipulation like inpainting and style transfer.

The release was genuinely open: a demo site, three model checkpoints on Hugging Face and inference code went out on the last day of 2023 for the research community. That openness is the main reason it still matters, even though the team has not announced follow-up versions.

Specs

Context window

Short audio clips (seconds-scale)

Pricing

Free (open weights)

Free quota

Unlimited (self-hosted)

API access

No hosted API; self-host via Hugging Face checkpoints

Weights

Open weights

Released

Not disclosed

Benchmarks

Objective and subjective evals show gains over prior TTA with limited data and compute (provider-reported).

Provider-reported unless a source says otherwise. Benchmarks are marketing-adjacent by nature — treat them as directional.

Use cases

text-to-audiosound-effectsaudio-editingresearch

Trust score — 7/10

/10 trustVerified

Scored 7/10 (verified tier, metrics-v1). Evidence: published paper, three open checkpoints on Hugging Face, public inference code and demo. Strengths: novel T2I-to-TTA transfer idea, genuinely open release, audio manipulation notebooks. Docked for: (1) evaluations are provider-reported; (2) quiet since release, no follow-up versions. License: open weights and code for research; check repo license before commercial use.

How the trust score is built

Full methodology →
Adoption · stars/downloads, log-scaled vs peers
6.0/10
Maintenance · recency, releases, issue load
6.0/10
Openness · license + docs
9.0/10
Manual review · sanity read + red-flag check
7.0/10
  • · Paper arXiv 2401.01044; released Dec 31, 2023
  • · Adapts text-to-image diffusion frameworks to text-to-audio
  • · 3 checkpoints on Hugging Face plus inference code
  • · Audio manipulation notebooks (inpainting, style transfer)
  • · No logo staged (research project, no brand mark)

Sources