How Visko streams real-time world models on Google Cloud TPUs

Visko, an AI startup focused on interactive world models, collaborated with Google Cloud TPU engineers to deploy its Orbis Live World Model on Cloud TPUs. The project aimed to transition from a PyTorch prototype to a production-ready pipeline capable of real-time streaming within a single month. The challenge centered on meeting a strict real-time clock, where every video and audio chunk had to render faster than playback speed to avoid freezing the live stream.

The Orbis pipeline combines video generation, super-resolution upscaling to 1664x936 at 18 fps, and synchronized audio generation on a single Cloud TPU slice. By chaining these models sequentially in BF16 precision, the team achieved end-to-end generation at 1.07x playback speed. This approach eliminated the need for batch processing, as each live stream operates as a unique, sequential session with no tolerance for latency.

To optimize performance, Visko’s engineering team leveraged three open-source tools: JAX for high-performance array operations, torchax for PyTorch interoperability on TPUs, and Pallas for custom TPU kernel programming. These tools enabled rapid iteration while maintaining bare-metal accelerator speed, reducing the total per-chunk latency to meet real-time demands.

The deployment demonstrates how startups can scale interactive generative AI workloads on Google Cloud’s AI Hypercomputer. By co-engineering the pipeline with Google Cloud TPU engineers, Visko proved that lean teams can achieve low-latency, continuous streaming without sacrificing quality or cost efficiency, unlocking new possibilities in interactive entertainment and beyond.