nemotron-halloween
A Halloween installation features a grumpy talking garage door powered by NVIDIA’s Nemotron Voice Agent, using local models to respond to trick-or-treaters with pirate complaints and a fiery grin.
A Halloween-themed interactive installation named MonsterHouse uses a wooden garage door rigged with a fiery mouth and animated facial expressions to engage trick-or-treaters. The project leverages NVIDIA’s Nemotron Voice Agent, Nemotron ASR, Nemotron 3.5 Bolt E4B 4GB, Magpie TTS Zeroshot, and Audio2Face-3D to create a self-contained system running on a single workstation.
The character’s design prioritized a menacing yet expressive appearance, with jagged wooden lips and animated flames, built in Autodesk Maya and exported to Unreal Engine for visual effects. A custom animation rig was developed to enable Audio2Face-3D to drive the facial expressions, resulting in a surprisingly economical monster without eyes, eyebrows, or blinking.
NVIDIA’s Nemotron Voice Agent provided the conversational foundation, allowing the house to listen, respond, and speak using local models optimized for low compute and memory. The Bolt E4B model, chosen for its small footprint and conversational capability, powered the dialogue pipeline, while Magpie TTS Zeroshot generated the grumpy pirate voice from short reference samples.
The system splits processing across two GPUs: an RTX PRO 6000 Blackwell Max-Q handles language and voice models, while an RTX PRO 4000 Blackwell runs Unreal Engine graphics and Audio2Face-3D. Sync adjustments ensure audio, animation, and effects align to maintain a convincing performance, with the house gradually softening to polite visitors.