Now generally available: GKE dynamic slicing for higher resilience, and efficiency for TPUs

Google Kubernetes Engine (GKE) now supports dynamic slicing for TPUs, decoupling logical slice formation from physical nodepool provisioning to address inefficiencies in static configurations. Previously, TPU workloads required fixed nodepools for each slice, leading to slow startup times and complex manual adjustments when workloads changed. Dynamic slicing eliminates the need to delete and recreate nodepools, enabling faster job initialization and reducing operational overhead for large-scale AI training and inference tasks.

Dynamic slicing introduces two methods: super-slicing and sub-slicing. Super-slicing uses TPU optical circuit switching (OCS) to dynamically connect multiple 16-VM TPU cubes into larger slices tailored to workload requirements, improving fleet utilization. Sub-slicing partitions a single 16-VM cube into smaller, isolated slices on demand, allowing concurrent workloads with different topology needs to share the same physical resources without interference.

The feature reduces job startup latency by up to five times and accelerates workload recovery by up to 4.5 times by isolating failures to smaller TPU cubes and swapping unhealthy units within minutes. It also enables partial utilization of nodepools with faulty hosts, maximizing resource efficiency and reducing idle capacity across large-scale AI deployments.

To implement dynamic slicing, administrators must pre-provision TPU capacity into 16-VM nodepools and enable the slice controller in GKE. Workloads request dynamic slices via Kubernetes annotations, triggering automatic slice formation at runtime. The scheduler then deploys workloads onto the newly created slices, which are automatically decommissioned upon completion, returning resources to the shared pool.