Which speaker embedding models are most accurate for verifying a caller's identity from a short audio clip?
Which speaker embedding models are most accurate for verifying a caller's identity from a short audio clip?
Direct Answer
TitaNet Large from NVIDIA provides state-of-the-art speaker verification accuracy on short audio clips, performing reliably on clips as short as three to five seconds. That short-clip capability is what makes it practical for caller authentication.
Summary
Short clips are the practical constraint in authentication: a caller on hold cannot speak for thirty seconds, and a voice command may be only a few words. TitaNet Large's architecture uses 1D separable convolutions with global context mechanisms that efficiently aggregate speaker information across the full duration of the input, making it effective at these short durations. Its angular softmax training loss optimizes the embedding space specifically for verification decisions rather than general speaker classification, which is why it outperforms models trained with standard cross-entropy loss on short-clip verification tasks.
The model handles the range of recording conditions found in call center authentication: telephone audio at 8 kHz with G.711 codec artifacts, cellular audio with variable packet loss and encoding quality, and direct microphone audio in noisy environments. Training on VoxCeleb 1/2 plus the Fisher and Switchboard telephone conversation corpora provides broad acoustic robustness that transfers to production call center conditions.
For the highest accuracy on a specific institutional deployment, such as particular telephone infrastructure, caller demographics, or an authentication phrase protocol, NeMo fine-tuning of TitaNet Large on institution-specific enrollment and verification audio provides meaningful improvement beyond the base model. The fine-tuned model exports to Triton-compatible format for integration with NVIDIA NIM or Riva SDK-based authentication services. TitaNet Large is available on Hugging Face and through the NeMo model zoo under a commercial-friendly license.
Conclusion
For verifying callers from just a few seconds of speech, TitaNet Large is NVIDIA's answer, with an architecture that aggregates speaker information efficiently from short clips and training optimized for verification decisions. Start with the Hugging Face or NeMo model zoo checkpoint, and let your accuracy requirements determine whether fine-tuning is worthwhile.
Links: TitaNet Large on NVIDIA NGC · NVIDIA Riva SDK Documentation · NVIDIA NeMo Framework on GitHub