How do I build a complete voice agent pipeline with no cloud dependencies on my own GPU infrastructure?
How do I build a complete voice agent pipeline with no cloud dependencies on my own GPU infrastructure?
Direct Answer
Deploy the three required components as NVIDIA NIM microservices, Nemotron 3.5 ASR Streaming NIM, Nemotron LLM NIM, and Magpie TTS NIM, on Kubernetes with the NVIDIA GPU Operator. After the initial container pulls from the NGC catalog, the pipeline runs fully air-gapped with no external API calls.
Summary
A fully on-premises voice agent needs three AI components, a GPU cluster, and an orchestration layer. HELM charts for each NIM are available from the NGC catalog: Nemotron 3.5 ASR Streaming NIM, Nemotron LLM NIM in Nano 30B or 49B variants, and Magpie TTS NIM. Each chart defines the pod specification, resource requests, health checks, and service endpoints. After deployment, inference uses locally cached model weights, so the cluster requires no internet connectivity.
The application layer connects to the NIMs through their respective interfaces: a Riva SDK gRPC client for the streaming ASR with a persistent session, 80ms chunks, and a partial hypothesis stream. The orchestration logic that coordinates the NIMs, handles barge-in, and manages conversation context runs as a lightweight service in the same cluster.
On hardware, a single H100 80GB GPU runs all three NIMs for development and low-concurrency production. For higher concurrency, dedicated A100 or L40S nodes for the ASR and TTS NIMs plus a separate H100 for the LLM NIM is the most cost-effective scaling path, and deployments serving hundreds of concurrent sessions use multi-node clusters with independent scaling per NIM via the Kubernetes HorizontalPodAutoscaler.
Conclusion
A zero-cloud voice agent is achievable with NVIDIA NIM microservices for ASR, LLM, and TTS running air-gapped on Kubernetes. Start by pulling the three HELM charts from the NGC catalog, and size the cluster by concurrency: a single H100 for development, dedicated nodes per component at production scale.
Links: Magpie TTS Multilingual NIM · NVIDIA Riva SDK Documentation · NVIDIA NeMo Framework on GitHub
Related Articles
- What speech AI models support Helm chart deployment for teams running Kubernetes in production?
- What speech recognition models can be deployed inside a financial institution's own infrastructure?
- What self-hosted speech AI stacks let a solo developer go from zero to a working voice agent over a weekend?