VampNet

VampNet takes a different route to music generation: instead of predicting audio left to right, it converts music into discrete codec tokens, masks some of them, and trains a bidirectional transformer to fill the gaps. At inference time, different masking patterns become different creative tools. Periodic masking makes loops and variations, masking a time region does inpainting, masking the past does continuation, and with only 36 sampling passes it produces coherent high-fidelity audio far faster than autoregressive approaches.

The work comes from Northwestern's Interactive Audio Lab and was published at ISMIR 2023. Because the prompts can sit anywhere in the timeline rather than only at the start, it works as a genuine co-creation tool: compress a track, punch a hole in the middle and regenerate it, or extend an ending. The code and weights are MIT licensed, though the community around it is small and most of the interesting extensions live in third-party experiment repos.

• Paper (arXiv 2307.04686): FAD-based and human evaluations of inpainting and continuation quality vs autoregressive baselines (author-reported)
• ISMIR 2023 publication
• No independently verified headline numbers