Magenta DDSP
Google Magenta's differentiable DSP library: neural nets that control real synthesizers, filters, and waveshapers for timbre transfer.
About
DDSP (Differentiable Digital Signal Processing) flips the usual neural-audio setup: instead of predicting raw waveforms, the network predicts the parameters of classic DSP building blocks (additive synthesizers, filters, waveshapers), which then render the audio. Because the synthesis path is interpretable signal processing, models train on far less data, run as tiny real-time networks, and give creators knobs they can actually understand.
Published at ICLR 2020 (Engel et al.), it powers Magenta's timbre-transfer demos (hum into a mic, hear a violin back) and real-time plugins like ddsp-vst, and it still anchors third-party neural instrument work. The library is Apache-2.0, ships as a pip package with TF.js builds, and the repo is actively maintained.
Specs
Context window
Pricing
Free quota
API access
Weights
Released
Benchmarks
No standard listener leaderboards; reconstruction-quality metrics in the ICLR 2020 paper (arXiv 2001.04643) rather than consumer benchmarks
Provider-reported unless a source says otherwise. Benchmarks are marketing-adjacent by nature — treat them as directional.
Use cases
Trust score — 8/10
Scored 8/10 (verified tier, metrics-v1). Evidence: ICLR 2020 paper (arXiv 2001.04643) from Google Magenta authors; official repo github.com/magenta/ddsp confirmed Apache-2.0 via multiple third-party attributions; repo actively maintained (recent commits); ddsp-vst plugin and ddsp-guitar-infer show real downstream reuse. Strengths: landmark hybrid neural/DSP approach, permissive license, data-efficient, real-time capable, maintained. Docked for: (1) it is a toolkit, not a ready-made generator — you train your own models; (2) pretrained checkpoint licensing is less clear than the code license. License: Apache-2.0 (code).
How the trust score is built
Full methodology →- Adoption · stars/downloads, log-scaled vs peers
- 8.0/10
- Maintenance · recency, releases, issue load
- 7.0/10
- Openness · license + docs
- 9.0/10
- Manual review · sanity read + red-flag check
- 8.0/10
- · ICLR 2020 (arXiv 2001.04643), Google Magenta authors
- · Repo github.com/magenta/ddsp actively maintained
- · ddsp-vst plugin and ddsp-guitar-infer show downstream reuse