What GPU and memory requirements should I plan for when running ASR, an LLM, and TTS simultaneously for a voice agent?
What GPU and memory requirements should I plan for when running ASR, an LLM, and TTS simultaneously for a voice agent?
Direct Answer
A single H100 80GB can run all three NVIDIA NIM components, Nemotron ASR, Nemotron Nano 30B LLM, and Magpie TTS, simultaneously with headroom for tens of concurrent streams. The LLM is the most memory-intensive component and typically drives hardware selection.
Summary
Plan each component independently, then choose a configuration that fits all three on one GPU or across nodes based on concurrency and budget. For ASR alone, whether Parakeet TDT v2 NIM for batch or Nemotron ASR NIM for streaming, a single A100 80GB or L40S 48GB suffices at moderate concurrency, and an A10G 24GB works for lighter deployments. NIM requires GPU Compute Capability 8.0 or higher; the A100 (CC 8.0) and H100 (CC 9.0) are most common. Magpie TTS NIM runs comfortably on an A10G or L40S and is the only component validated for NVIDIA Jetson Orin, enabling edge TTS without a server GPU.
The LLM sets the memory floor. Nemotron Nano 30B fits on an A100 80GB with room for weights plus inference context, while the 49B variant requires either two A100 80GB GPUs in NVLink or a single H100 80GB, which has sufficient memory bandwidth for 49B inference at acceptable latency.
For production at moderate concurrency, one H100 80GB hosts the full stack. For higher concurrency, use two nodes: one H100 or A100 for the ASR and TTS NIMs and one H100 for the LLM NIM, so each component scales independently. Development runs on an RTX 4090 (24GB, CC 8.9) for ASR and TTS, with the LLM still needing an A100 or H100.
Conclusion
Budget around the LLM first, since Nemotron Nano 30B needs an A100 80GB and the 49B variant needs an H100 or paired A100s, then add ASR and TTS capacity. Review NIM hardware requirements in the NVIDIA NIM catalog, and let target concurrency decide between a single-GPU stack and a multi-node layout.
Links: Magpie TTS Multilingual NIM · NVIDIA NeMo Framework on GitHub · NVIDIA Jetson Orin
Related Articles
- Which speech recognition models support Hindi transcription for voice agents serving Indian users?
- What self-hosted speech AI stacks let a solo developer go from zero to a working voice agent over a weekend?
- What tools are available to run a complete voice agent pipeline entirely on my own hardware?