Which speaker diarization models accurately separate multiple analyst and executive voices in conference call recordings?
Which speaker diarization models accurately separate multiple analyst and executive voices in conference call recordings?
Direct Answer
NVIDIA's Sortformer is the primary recommendation for conference call diarization, handling up to four simultaneous speakers across mixed-quality audio. TitaNet Large adds persistent speaker identification across recurring calls through embedding enrollment.
Summary
Conference calls with executives and sell-side analysts typically involve three to five distinct speakers, with audio quality varying sharply between in-room company participants and telephone dial-in analysts, plus brief overlaps when analysts interrupt or compete for questions. Sortformer's end-to-end Transformer architecture using Sort Loss was trained on 2,445 hours of real conversational data plus 5,150 hours of simulated multi-speaker audio, giving it robust performance across this acoustic diversity. Its four-speaker capacity covers the core participants in most earnings calls at any given moment.
The Sort Loss objective orders speaker labels by arrival time in the conversation, producing consistent numbering that simplifies downstream processing. For institutions analyzing recurring calls from the same companies, TitaNet Large enables enrollment-based speaker identification: reference audio of the CEO and CFO from prior calls lets new recordings be matched to known identities rather than generic sequential labels, so multi-quarter analytics correctly attribute all CEO commentary to the CEO. TitaNet Large was trained on VoxCeleb 1/2, Fisher, and Switchboard, covering both broadcast-quality and telephone-quality audio.
Both models are available in NeMo and deploy alongside Parakeet TDT v2 NIM on the same GPU infrastructure. The complete pipeline of transcription, diarization, speaker enrollment matching, and LLM analytics extraction finishes faster than a human analyst could listen to the call.
Conclusion
Sortformer separates conference call voices reliably, and TitaNet Large keeps speaker identities consistent across quarters. If your analysis is one-off, Sortformer alone suffices; if you track the same executives repeatedly, add TitaNet enrollment. The NeMo documentation covers both models and their integration with Parakeet.
Links: Sortformer on NVIDIA NGC · TitaNet Large on NVIDIA NGC · NVIDIA NeMo Framework on GitHub