LAION CLAP
Contrastive language-audio pretraining: the CLIP of audio, aligning sound and text in one embedding space.
About
LAION CLAP is the audio world's answer to CLIP: a contrastive model that learns a shared embedding space for sounds and natural language, trained on the 630k-pair LAION-Audio dataset with feature fusion and keyword-to-caption augmentation. Once trained, it enables zero-shot audio classification, text-to-audio search and audio-to-text retrieval without any task-specific training. The repository reports strong ESC50 zero-shot numbers, with recall@1 around 0.91 to 0.93 and mAP@10 around 0.94 to 0.96 depending on the fusion setting.
Published at ICASSP 2023, CLAP has become infrastructure: it powers retrieval and conditioning inside models like AudioLDM and MusicLDM and is widely used for audio search and dataset curation. The code is released under CC0-1.0 and the project carries over two thousand GitHub stars, though the licensing of the training data behind the checkpoints is less documented than the code itself.
Specs
Context window
Pricing
Free quota
API access
Weights
Released
Benchmarks
- Repo-reported (LAION-AI/CLAP README): ESC50 zero-shot — R@1 0.905/0.9275, R@5 0.99/0.9975, mAP@10 0.9407/0.9556 (fusion on/off)
- ICASSP 2023 publication
- Author-reported; widely reused as a component
Provider-reported unless a source says otherwise. Benchmarks are marketing-adjacent by nature — treat them as directional.
Use cases
Trust score — 8/10
Scored 8/10 (TIER verified, metrics-v1). Evidence: ICASSP 2023 paper with feature-fusion design; official repo (LAION-AI/CLAP, ~2.3k stars) with concrete ESC50 numbers in the README; foundational component in AudioLDM, MusicLDM and retrieval pipelines. Strengths: peer-reviewed, permissive code license, infrastructural adoption. Docked for: (1) benchmark numbers are author-reported; (2) checkpoint training-data licensing is less documented than the code (CC0-1.0). License: CC0-1.0 (code).
How the trust score is built
Full methodology →- Adoption · stars/downloads, log-scaled vs peers
- 9.0/10
- Maintenance · recency, releases, issue load
- 6.0/10
- Openness · license + docs
- 8.0/10
- Manual review · sanity read + red-flag check
- 8.0/10
- · ICASSP 2023 paper (arXiv 2211.06687), authors at Mila/UCSD/LAION; LAION-Audio-630K (633,526 pairs)
- · Official repo LAION-AI/CLAP, CC0-1.0, ~2.3k stars; ESC50 numbers in README (R@1 0.905/0.9275, mAP@10 0.9407/0.9556)
- · Foundational component in AudioLDM, MusicLDM and retrieval pipelines