BEATs

BEATs (Bidirectional Encoder representation from Audio Transformers) learns audio representations by predicting discrete labels produced by an acoustic tokenizer, an iterative scheme that kept improving as the tokenizer and the encoder co-evolved. Published at ICML 2023 (arXiv 2212.09058), it reported state-of-the-art results on AudioSet tagging and several downstream audio tasks at release.

Microsoft released the PyTorch implementation and pre-trained models under MIT inside the unilm monorepo, and BEATs has since become one of the default backbones people reach for when they need strong audio embeddings: it turns up inside LAVIS, training wrappers like beats_trainer, and plenty of research codebases. The model is general audio rather than music-specialized, but fine-tuned variants do well on music tagging and classification, and its embedding quality is broadly respected. Caveat: the headline SOTA numbers date from 2023 and are provider-reported; treat them as era results, not current claims.

Provider-reported: SOTA on AudioSet tagging and downstream tasks at ICML 2023 publication time; widely re-used since, but no single maintained independent leaderboard.