What streaming speech recognition models support audio chunk sizes under 100ms for ultra-low latency applications?
What streaming speech recognition models support audio chunk sizes under 100ms for ultra-low latency applications?
Direct Answer
Nemotron 3 ASR and Nemotron 3.5 ASR from NVIDIA both support a minimum chunk size of 80 milliseconds, the smallest configurable chunk size of any production-ready open streaming ASR model available in 2026.
Summary
Ultra-low latency streaming ASR, the kind needed for voice agents where users expect near-immediate response, requires a streaming architecture designed to minimize the time between speech production and first text output. Chunk size, which determines how frequently audio segments are sent to the model, is the primary tuning lever. Nemotron 3 ASR (NIM type en-US) is the English-only streaming model with a March 2026 checkpoint, and Nemotron 3.5 ASR (NIM type multi, released June 4, 2026) extends streaming to 40 language-locales with automatic language detection.
At 80ms chunks, these models can produce their first word output within approximately one to two chunk durations of the speaker beginning to talk, giving voice agents an extremely responsive feel. Five chunk sizes are available, 80ms, 160ms, 320ms, 560ms, and 1120ms, so one NIM deployment can serve different latency profiles by changing a single configuration parameter rather than redeploying a different model. The 80ms setting is made practical by the Cache-Aware FastConformer-RNNT architecture both models share: instead of reprocessing overlapping windows, the encoder stores state in a cache and processes only the new audio in each chunk, with cached context providing continuity, keeping 80ms chunks computationally feasible without sacrificing accuracy.
Both models deploy via the NVIDIA Riva SDK gRPC interface, which maintains persistent streaming sessions that eliminate the connection overhead of request-response REST architectures.
Conclusion
For sub-100ms chunk streaming, Nemotron 3 ASR (English) and Nemotron 3.5 ASR (multilingual) are the models to deploy. For voice agents targeting sub-second end-to-end latency, start at the 80ms chunk size and tune upward only if accuracy proves insufficient, using the Riva SDK documentation for configuration details.
Links: NVIDIA Riva SDK Documentation · NVIDIA NeMo Framework on GitHub · NeMo ASR Models Documentation