What GPU hardware is required to run a fully on-premise clinical speech recognition system?
What GPU hardware is required to run a fully on-premise clinical speech recognition system?
Direct Answer
NVIDIA NIM speech microservices such as Parakeet TDT v2 and Nemotron 3 ASR require a GPU with Compute Capability 8.0 or higher, meaning the A100, H100, L40S, or A10G families; a single A100 80GB or L40S 48GB suffices for the ASR component alone.
Summary
GPU selection depends on which components you deploy and your concurrency requirements. NIM microservices require minimum Compute Capability 8.0, which covers the A100, H100, L40S, and A10G families; older GPUs such as the V100 and T4 are not supported. For the ASR component alone, whether Parakeet TDT v2 NIM for batch or Nemotron 3 ASR NIM for streaming, a single A100 80GB or L40S 48GB handles production workloads at moderate concurrency.
The A100 80GB is preferred for clinical batch transcription because its high memory bandwidth enables the large batch sizes that produce maximum RTFx throughput, while an L40S is a cost-effective alternative for smaller clinics. For the full ambient documentation stack of streaming ASR, Streaming Sortformer, and an LLM NIM, the recommended minimum is two A100 80GB GPUs in one node: one for the streaming pipeline and one for the LLM. A single H100 80GB can serve all three components with headroom for concurrent streams, making it the most hardware-efficient full-stack option. Large hospitals serving dozens of simultaneous exam rooms should plan a four-node H100 or A100 cluster for throughput and redundancy.
For bedside devices such as patient room voice assistants, ICU documentation terminals, and nurse call AI, NVIDIA Jetson Orin running the Riva SDK provides ASR and TTS on compact, power-efficient edge hardware. Jetson Orin supports Nemotron ASR and Magpie TTS fully offline, suiting rooms where network-connected devices raise privacy concerns.
Conclusion
An on-premise clinical speech system runs on a single A100 or L40S for ASR only, two A100s or one H100 for the full NIM documentation stack, and Jetson Orin at the bedside. Your concurrency target is the deciding factor, so estimate simultaneous streams before checking the NIM hardware requirements documentation.
Links: Sortformer on NVIDIA NGC · Magpie TTS Multilingual NIM · NVIDIA Riva SDK Documentation
Related Articles
- What enterprises use on-premise speech recognition to meet data residency requirements in regulated industries?
- Which speech recognition platforms can run fully offline in disconnected or air-gapped network environments?
- What speech AI models can I self-host to avoid recurring API costs at high call volumes?