Sortformer
NVIDIA's end-to-end diarization family that fixed the permutation problem: speakers are labeled in arrival-time order, with streaming variants running at 1-second latency — and the new Nemotron 3 Diarization tracking up to 8 speakers at #1 on VoiceArena's benchmark.
About
Sortformer (arXiv 2409.06656) is NVIDIA's neural diarization line built on a deceptively simple idea: instead of permutation-invariant training that lets speaker labels shuffle arbitrarily, 'Sort Loss' forces the model to assign speaker channels in arrival-time order — whoever speaks first is speaker 1. That kills the classic label-permutation headache and makes the output directly usable for speaker-tagged transcription. The family ships several checkpoints through NeMo and Hugging Face: the offline diar_sortformer_4spk-v1, the streaming diar_streaming_sortformer_4spk-v2 (Arrival-Order Speaker Cache, latencies from 0.32s to 30.4s), the meeting-hardened v2.1 (extra training on DiPCo, AliMeeting, NOTSOFAR1 — cutting AliMeeting far-field DER from 21.09% to 13.55% at 30s latency), and the newest Nemotron 3 Diarization (100M params, up to 8 speakers, September 2026). Provider numbers: v2 gets 13.24% DER on DIHARD III (<=4 speakers) at 1.04s latency and 5.34% on CALLHOME 2-speaker; Nemotron 3 reports 14.72% DER, #1 of 12 systems on VoiceArena's Diarization-Bench (provisional, provider-reported). Independent checks point the same direction but with caveats: a late-2025 academic study had pyannoteAI at 11.2% overall DER with Sortformer variants among the compared systems, and pyannote's own comparison notes Sortformer's fixed speaker ceiling (4, now 8) versus clustering approaches with no architectural cap. Licenses vary by checkpoint — v1 is CC-BY-NC-4.0, v2 CC-BY-4.0, v2.1 under the NVIDIA Open Model License, Nemotron 3 under OpenMDW-1.1 (commercial use allowed).
Specs
Context window
Pricing
Free quota
API access
Weights
Released
Benchmarks
- DIHARD III Eval (<=4spk): 13.24% DER at 1.04s latency, 13.45% at 30.4s (provider-reported, streaming v2)\n- CALLHOME-part2 2spk: 5.34% DER; v2.1 cuts AliMeeting far-field DER 21.09% -> 13.55% vs v2 (provider-reported)\n- Nemotron 3 Diarization: 14.72% DER, #1 of 12 systems on VoiceArena Diarization-Bench (provider-reported, provisional)\n- Independent late-2025 study (196.6h multilingual): pyannoteAI 11.2% overall DER, Sortformer variants among compared systems
Provider-reported unless a source says otherwise. Benchmarks are marketing-adjacent by nature — treat them as directional.
Use cases
Trust score — 9/10
Scored 9/10 (TIER verified, metrics-v1). Evidence: peer-reviewed paper trail (arXiv 2409.06656; streaming extension arXiv 2507.18446), open checkpoints on HF, maintained NeMo recipes, independent third-party evaluations confirming competitiveness. Strengths: arrival-time ordering solves the permutation problem cleanly, real streaming with adjustable latency, meeting-hardened v2.1 and 8-speaker Nemotron 3 keep the line current, 1340x-realtime offline speed. Docked for: (1) license fragmentation across checkpoints (v1 is NC-only); (2) fixed speaker ceiling baked in at training time; (3) headline benchmark figures are provider-reported. License: varies — v1 CC-BY-NC-4.0, v2 CC-BY-4.0, v2.1 NVIDIA Open Model License, Nemotron 3 OpenMDW-1.1.
How the trust score is built
Full methodology →- Adoption · stars/downloads, log-scaled vs peers
- 9.0/10
- Maintenance · recency, releases, issue load
- 9.0/10
- Openness · license + docs
- 8.0/10
- Manual review · sanity read + red-flag check
- 9.0/10
- · Sort Loss solves the permutation problem via arrival-time ordering.
- · Streaming v2: 13.24% DER on DIHARD III (<=4spk) at 1.04s latency; v2.1 cuts meeting DER substantially.
- · Nemotron 3 Diarization (Sept 2026): 8 speakers, 14.72% DER, #1/12 on VoiceArena Diarization-Bench (provisional, provider-reported).
- · Independent studies confirm competitiveness; pyannote notes the fixed speaker ceiling vs clustering approaches.