ElevenLabs on AI Gateway
Vercel’s AI Gateway now integrates 14 ElevenLabs audio models with a unified API, enabling real-time transcription, speech, music, and sound effects generation for applications.
Vercel and ElevenLabs have launched a partnership that adds 14 ElevenLabs models to AI Gateway, a unified API for managing AI model calls. The integration covers speech-to-text, real-time voice agents, music generation, sound effects, and multi-voice dialogue. Developers can access these models through a single API key, streamlining workflows that previously required separate integrations. The announcement includes a 50% discount on ElevenLabs models via AI Gateway for the first 14 days, ending November 9th.
The partnership enables applications to transcribe user speech, process it with a language model, and generate spoken responses in real time. ElevenLabs’ Scribe v2 Realtime converts speech to text as the user speaks, while Eleven Flash v2.5 streams the language model’s response aloud before completion. This reduces latency, allowing users to hear answers as they are generated rather than waiting for the full response.
Realtime voice agents like Eleven Realtime v1 Max and v1 Mini handle full conversations over a single WebSocket session, eliminating the need for separate transcription or speech steps. These models support semantic voice activity detection to determine when a user has finished speaking, though they do not support server-side VAD. The browser manages the WebSocket connection, microphone capture, and audio playback via the useRealtime hook from @ai-sdk/react.
AI Gateway consolidates usage tracking, cost management, and performance optimizations for all ElevenLabs models alongside other AI services. Features include built-in custom reporting, Zero Data Retention support, API key budgets, and routing rules. The unified API simplifies deployment by handling retries, failover, and uptime improvements, reducing the complexity of integrating multiple audio models.