Build Low-Latency Multilingual Voice Agents: Open Weights & Full Deployment Control with NVIDIA Magpie TTS

NVIDIA introduced Magpie Multilingual TTS, an open-weight text-to-speech model designed for low-latency voice applications. The model now supports 12 languages, including Modern Standard Arabic, Korean, and Brazilian Portuguese, expanding its multilingual capabilities. It is optimized for real-time speech generation, allowing developers to deploy it within their own infrastructure for greater control over latency and customization.

The release improves speech quality across existing languages through updated training data and model enhancements. Magpie supports male and female voices in each language via a shared multilingual speaker representation. It also introduces expanded code-switching support for Hindi and Japanese, facilitated by IPA-based grapheme-to-phoneme processing and custom pronunciation dictionaries, enabling accurate pronunciation of mixed-language content and technical terms.

Magpie TTS is distributed as an open Hugging Face checkpoint for research and fine-tuning, alongside an NVIDIA NIM serving stack for production deployments. Both versions run on-premises, allowing direct performance benchmarking and tuning. The model achieves Time to First Audio (TTFA) as low as 32ms on NVIDIA B200 GPUs, enabling end-to-end voice pipelines to stay within the sub-200ms latency threshold required for natural conversation.

The architecture includes frame stacking and a local transformer to reduce inference time while preserving speech quality. Frame stacking doubles audio frame prediction per decoding step, cutting iterations in half, while the local transformer refines generated audio to maintain high quality. These optimizations support real-time multilingual voice agents across customer support, healthcare, enterprise assistants, and translation systems.