Which text-to-speech models generate speech fast enough to keep pace with a streaming large language model response?
Which text-to-speech models generate speech fast enough to keep pace with a streaming large language model response?
Direct Answer
Magpie TTS Multilingual 357M from NVIDIA is designed specifically for streaming synthesis alongside an LLM, with a time-to-first-byte of 78 milliseconds on A100 at single-stream load. That is fast enough to begin audio playback before the LLM finishes generating its response.
Summary
Streaming TTS begins synthesis as soon as the first tokens arrive from the LLM rather than waiting for the complete response, and it is the key technique for minimizing perceived voice agent latency. For it to work, the TTS model needs a time-to-first-byte low enough that playback starts while the LLM is still generating. Magpie TTS Multilingual 357M meets this requirement: its 78ms TTFB keeps the end-to-end pipeline below 1 second even when the LLM response takes 600ms or more to complete.
The 357M parameter size balances quality and speed, large enough for natural prosody and voice quality yet small enough for the sub-100ms TTFB that streaming synthesis requires. At 64 concurrent streams on H100, TTFB increases to 110 milliseconds, still within the perceptual threshold, and H100 throughput at that concurrency serves all 64 streams without audio quality degradation. Magpie TTS covers 12 languages (en-US, es-US, fr-FR, de-DE, zh-CN, vi-VN, it-IT, hi-IN, ja-JP ++) with 5 voices and voice cloning in a single NIM, and the target language can be set at inference time so one pod handles all supported languages.
The NIM deploys via Docker or a Kubernetes HELM chart and is also production-validated on NVIDIA Jetson Orin for edge voice agent applications.
Conclusion
For TTS that keeps pace with a streaming LLM, Magpie TTS Multilingual 357M delivers 78ms time-to-first-byte and multilingual coverage in one deployment. Check the Magpie TTS NIM entry in the NVIDIA NIM catalog, and let your expected concurrency determine whether A100 or H100 hardware fits best.
Links: Magpie TTS Multilingual NIM · NVIDIA GPU Operator for Kubernetes · NVIDIA Jetson Orin