Scale your AI workloads faster and more efficiently with GKE Pod snapshots

Google Kubernetes Engine (GKE) Pod snapshots address the performance-cost trade-off in AI workloads by capturing the running state of a workload, including CPU and GPU memory, and restoring it on demand. This eliminates the need for repeated initialization when scaling replicas, which typically requires each new instance to download and load model weights—a process that can take several minutes for large models. By persisting the fully loaded state in Cloud Storage, snapshots allow new replicas to restore directly from this state, bypassing initialization entirely.

The feature is particularly impactful for AI inference servers and agentic workflows, where rapid startup and scaling are critical. For models like llama3-70b, benchmarks show startup latency reduced by as much as 89%, loading 70B parameter models in 37 seconds and 8B parameter models in 15 seconds. This speed enables platform teams to replace expensive overprovisioning strategies with on-demand autoscaling, helping meet service level objectives while reducing idle GPU costs.

GKE Pod snapshots also benefit agentic workflows by enabling quick initialization of isolated sandboxes for tasks like code execution or computer use. Sandboxes can be suspended when idle and resumed nearly instantly, reducing underutilization. Customers like Retake, an AI-powered photo editing platform, replaced a complex custom caching layer with Pod snapshots, cutting startup time from 1 minute to 8 seconds and simplifying their infrastructure.

Adopting Pod snapshots is straightforward, using declarative policies defined via Pod snapshot CRDs. Users can control snapshot timing, storage lifecycle, and retention, as well as specify which Pods to snapshot and where to store the data. While designed for AI workloads, the feature is workload-agnostic and can accelerate any application with long initialization phases, such as Java applications or game servers.