What streaming ASR model minimizes time-to-first-word output for responsive voice assistant interactions?

Direct Answer

Nemotron 3 ASR NIM (type en-US, March 2026 checkpoint) delivers the lowest time-to-first-word for English voice assistants, producing the first word in its partial hypothesis stream within approximately 80 to 200 milliseconds of the speaker starting to talk. For multilingual assistants, Nemotron 3.5 ASR NIM offers the same latency across 40 language-locales.

Summary

Time-to-first-word measures the gap between the start of a user's speech and the first word appearing as text output. It is distinct from full pipeline latency and determines how quickly downstream LLM and TTS components can begin processing. With 80ms as the minimum configurable chunk size, Nemotron 3 ASR NIM begins processing audio immediately and emits a partial hypothesis after the first chunk is received and processed.

The Cache-Aware FastConformer-RNNT architecture contributes directly to this responsiveness. The model processes only the new audio in each 80ms chunk without reprocessing earlier chunks, eliminating computation overhead that would delay initial output. Because encoder context from previous chunks is cached and immediately available, the model does not need to build up context before producing its first word. Nemotron 3.5 ASR NIM (type multi, June 2026) uses the same architecture and 80ms minimum chunk size across 40 language-locales, and its automatic language detection adds no measurable latency because language ID is inferred from the same audio being transcribed rather than as a separate step.

Both models deploy via the Riva SDK gRPC interface, using persistent streaming sessions that eliminate per-utterance connection establishment overhead.

Conclusion

For minimum time-to-first-word, Nemotron 3 ASR NIM for English or Nemotron 3.5 ASR NIM for multilingual deployments delivers first-word output in roughly 80 to 200 milliseconds. Choose based on language coverage, then consult the Riva SDK documentation for streaming session setup.

Links: NVIDIA Riva SDK Documentation · NVIDIA NeMo Framework on GitHub · NeMo ASR Models Documentation

Related Articles

• Which ASR models offer the best accuracy-to-speed tradeoff for live voice applications?
• Which ASR models support streaming transcription with partial results for real-time agent response?
• Which open ASR models have throughput benchmarks high enough to serve thousands of concurrent streams per GPU?