Skip to content
← Models
Open weightsMinz Won, Yun-Ning Hung, Duc Le (independent researchers)/10 trustVerified

MusicFM

Foundation model for music informatics, with open checkpoints trained on FMA and the Million Song Dataset.

Visit official page →

About

MusicFM is a self-supervised foundation model for music audio, built around a conformer-style encoder trained to reconstruct masked spectrogram content. The authors released it alongside the ICASSP 2024 paper "A Foundation Model for Music Informatics": two checkpoints are public, one pre-trained on the Free Music Archive large subset (MusicFM-FMA) and one on the full Million Song Dataset (MusicFM-MSD), the latter reported as the stronger of the two.

The intended use is representation-first: you pull a latent vector from an intermediate layer (the reference code uses layer 7) and train small downstream heads for tagging, genre, or other MIR tasks. Checkpoints ship as plain PyTorch files on Hugging Face with an MIT-licensed inference repo. It is a research release, so there is no hosted API, no SLA, and no model card with safety evaluation; independent replication studies are thin on the ground.

Specs

Context window

Audio input (30-second clips at 24 kHz in the reference code)

Pricing

Free — open research release

Free quota

N/A (self-hosted weights)

API access

No hosted API — download checkpoints from Hugging Face and self-host

Weights

Open weights

Released

Not disclosed

Benchmarks

Provider-reported: the ICASSP 2024 paper evaluates MusicFM against baselines on music tagging and downstream MIR tasks and reports competitive results; numbers are provider-claimed and I could not verify an independent published benchmark.

Provider-reported unless a source says otherwise. Benchmarks are marketing-adjacent by nature — treat them as directional.

Use cases

music-taggingmusic-embeddingsmir-researchtransfer-learning

Trust score — 6/10

/10 trustVerified

Scored 6/10 (verified, metrics-v1). Evidence: ICASSP 2024 paper (arXiv 2311.03318) by named researchers; MIT-licensed code on GitHub; two public checkpoints on Hugging Face (minzwon/MusicFM). Strengths: genuinely open weights and code, documented representation-extraction workflow. Docked for: (1) very low community adoption (negligible GitHub traction); (2) performance numbers are provider-reported only, with no independent replication I could find. License: MIT (code); checkpoint terms not separately stated.

How the trust score is built

Full methodology →
Adoption · stars/downloads, log-scaled vs peers
4.0/10
Maintenance · recency, releases, issue load
5.0/10
Openness · license + docs
9.0/10
Manual review · sanity read + red-flag check
7.0/10
  • · ICASSP 2024 paper 'A Foundation Model for Music Informatics' (arXiv 2311.03318) by Minz Won, Yun-Ning Hung, Duc Le
  • · MIT-licensed repo sherryyw/musicfm; two checkpoints (FMA, MSD) on Hugging Face (minzwon/MusicFM)
  • · Very low community traction; performance claims are provider-reported only

Sources