Google unveils new TPU generation optimized for agents and state‑of‑the‑art Model Training

Google announced a new generation of Tensor Processing Units featuring two specialized chips — TPU 8t for large — scale, compute‑heavy training and TPU 8i for latency‑sensitive inference.

Google has introduced a new generation of Tensor Processing Units (TPUs) that includes two purpose — built chips intended to accelerate both frontier model training and agent workflows that run continuous, multi‑step reasoning and distributed action loops. The company says the new TPUs improve performance, memory capacity and energy efficiency to better match the distinct demands of training and inference workloads.

The TPU 8t targets massive, compute‑intensive training jobs. It increases compute density, memory capacity and bandwidth across large clusters to maximize raw scale and speed. According to Google, these changes deliver nearly three times the compute performance of the prior TPU generation and are intended to cut the time required to train frontier models “from months to weeks.

At scale, a single TPU 8t superpod can comprise 9,600 chips and provide two petabytes of shared high‑bandwidth memory, with double the interchip bandwidth of the previous generation. That configuration yields 121 ExaFlops of compute and lets the most complex models leverage a single, massive pool of memory. Google also says the architecture can scale almost linearly to clusters of up to a million chips in a single local deployment.

The TPU 8i is designed for the opposite end of the workload spectrum: latency‑sensitive inference. It emphasizes higher memory bandwidth to serve real‑time or near‑real‑time model inference, making it better suited to workloads that must respond quickly and coordinate actions across multiple models. Google positions TPU 8i as a complement to TPU 8t for agent pipelines that separate training and serving to extract greater efficiency.

Google argues that the rise of AI agents — systems that orchestrate multiple models in continuous decision and action loops — requires distinct hardware tuned for either training or inference to unlock substantial performance gains. By offering a pair of specialized chips, the new TPU generation aims to address those divergent needs and accelerate both the development of state‑of‑the‑art models and the low‑latency serving those models require.