3D-Speaker
Alibaba's multi-modal speaker toolkit: verification, recognition and diarization (audio-only or audio+video) with challenge-grade models like CAM++ and ERes2NetV2, all Apache 2.0.
About
3D-Speaker (modelscope/3D-Speaker, ICASSP 2025 toolkit paper, arXiv 2306.15354) is Alibaba DAMO Academy's open toolkit for speaker verification, recognition and diarization — notable for being one of the few open projects that does audio-visual diarization (face + voice) as well as audio-only. The model zoo is its calling card: CAM++ (7.2M params, 0.65% EER on VoxCeleb1-O), ERes2NetV2 (17.8M, 0.61% EER), ERes2Net-large (22.5M, 0.52% EER) — all competitive with the best published numbers. For diarization specifically, the repo's own benchmark table reports DER of 10.30% on Aishell-4 (vs 12.2% pyannote.audio, 11.7% DiariZen_WavLM), 11.75% on VoxConverse (vs 11.3% pyannote), 19.73% on Alimeeting (vs 24.4% pyannote), plus internal Chinese meeting sets where it leads comfortably. Those are provider-reported, but the head-to-head framing against named competitors is unusually transparent. It's also bundled with the large-scale 3D-Speaker corpus (multi-device, multi-distance, multi-dialect) for representation-disentanglement research. Everything is Apache 2.0, including models and corpus — the cleanest licensing story in this batch. Strong pick if your audio is Mandarin-heavy or you want audio-visual diarization without license anxiety.
Specs
Context window
Pricing
Free quota
API access
Weights
Released
Benchmarks
- Speaker verification: CAM++ 0.65% EER, ERes2NetV2 0.61% EER, ERes2Net-large 0.52% EER on VoxCeleb1-O (provider-reported)\n- Diarization DER: Aishell-4 10.30% (pyannote 12.2%, DiariZen 11.7%); VoxConverse 11.75% (pyannote 11.3%); Alimeeting 19.73% (pyannote 24.4%); AMI_SDM 21.76% (provider-reported head-to-head)\n- No independent replication found
Provider-reported unless a source says otherwise. Benchmarks are marketing-adjacent by nature — treat them as directional.
Use cases
Trust score — 8/10
Scored 8/10 (TIER verified, metrics-v1). Evidence: ICASSP 2025 peer-reviewed toolkit paper (arXiv 2306.15354); transparent head-to-head DER table against pyannote and DiariZen; maintained repo (pushed December 2025). Strengths: rare open audio-visual diarization, strong verification numbers, clean Apache 2.0 licensing across code+models+corpus, genuinely multilingual (Mandarin-strong). Docked for: (1) all benchmark figures provider-reported with no independent replication; (2) toolkit-style repo — more assembly required than a turnkey pipeline. License: Apache 2.0.
How the trust score is built
Full methodology →- Adoption · stars/downloads, log-scaled vs peers
- 7.0/10
- Maintenance · recency, releases, issue load
- 8.0/10
- Openness · license + docs
- 9.0/10
- Manual review · sanity read + red-flag check
- 8.0/10
- · ICASSP 2025 toolkit paper; rare open audio-visual diarization.
- · Transparent head-to-head DER table vs pyannote and DiariZen (provider-reported).
- · Mandarin-strong; bundled multi-device/multi-distance/multi-dialect corpus.