Skip to content
ISSUE 001·LIVE·20:20 IL
← Journal/2026-11-11·8 min·google cloud

Google TPU 8tx and 8i Max: confidential inference and million-token context get dedicated systems

Google announced TPU 8tx and TPU 8i Max on November 9, completing four eighth-generation TPU systems in one year: 8t for training, 8i for reasoning, 8tx for hardware-level confidential inference, and 8i Max for massive context windows. What each system does, the open software stack around them, and what is still unannounced.

By Harel Asaf·AI Builder·Tel Aviv

Google announced TPU 8tx and TPU 8i Max on November 9, bringing its eighth-generation TPU portfolio to four systems in a single year. The split tells you where AI infrastructure demand is heading: TPU 8tx is built for confidential inference, keeping data and model weights encrypted at the hardware level even while in use, and TPU 8i Max doubles the compute, SRAM and HBM bandwidth of TPU 8i by pairing two chips on one package, aimed at agent workloads reasoning across million-token contexts. Alongside the hardware, Google open-sourced TorchTPU (PyTorch code runs natively on TPUs, no rewrites), TPU Sync (a model-state layer it says cuts KV transfer latency and time to first token by over 90%), and an agent suite that automates TPU onboarding from inside Claude Code, Cursor and Antigravity. Pricing and availability dates for the two new systems were not in the announcement.

What launched

Per Google's announcement by Amin Vahdat, SVP and Chief Technologist for AI and Infrastructure:

  • TPU 8tx: optimized for secure and confidential inference and small-scale training. Built on the same silicon as TPU 8t with 5th Generation AMD EPYC CPU hosts, it keeps sensitive data and model weights protected and isolated even while actively processed. It uses the low-latency Boardfly network, delivering up to 2x lower latency than TPU 8t with what Google calls a near-zero performance penalty.
  • TPU 8i Max: two TPU 8i chips on a single package, doubling compute, SRAM and HBM bandwidth over TPU 8i, with twice the silicon in the existing rack footprint. It targets the memory wall that appears when many agents work across an entire codebase or hours of video at once.
  • The full eighth-generation lineup now maps to stages of the agentic lifecycle: 8t for scale-up training, 8i for low-latency reasoning, 8tx for confidential computing, and 8i Max for massive context windows and the highest compute density per rack.
  • Baseline numbers from April for this generation: TPU 8t offers up to 2.7x better training performance and TPU 8i up to 80% better inference price-performance versus the previous generation. Google says TPUs are now a business with nearly 100 customers, including frontier labs, trading firms, retailers and universities, and customers can deploy TPUs in their own data centers.

Why the specialization matters

The design philosophy Google describes is "building the computer around the chip": keep the silicon modular, then vary packaging, memory, networking and security around it to ship new systems in months instead of the multi-year cycle a new chip requires. That is how one generation produced four systems in a year.

The two new systems map onto the two constraints agentic workloads actually hit. First, context: agents reading whole repositories or long video are bottlenecked by memory bandwidth, not raw compute, so 8i Max scales both in lockstep. Second, trust: regulated enterprises and governments want frontier models and autonomous agents next to their most sensitive data, and normal encryption leaves data exposed in memory during processing. 8tx moves that protection into hardware. Because 8tx pairs training-grade compute with a fast inference network, Google says one system can fine-tune on private data, distill large models into smaller ones, and train agents with reinforcement learning without moving data between clusters.

The software stack is the real pitch

The hardware is only half the announcement. The other half is Google systematically removing the reasons developers avoided TPUs:

  • TorchTPU, open-sourced last month, runs PyTorch code and Hugging Face models natively on TPUs with no rewrites, alongside TPU backends for vLLM and SGLang.
  • A new open-source TPU agent suite, now in preview, plugs into coding tools including Claude Code, Cursor and Google Antigravity to automate TPU onboarding, diagnose bottlenecks and autotune custom kernels in plain language. The notable detail: Google's own onboarding agents live inside third-party coding assistants.
  • The open-source XLA Serving Engine lets teams train in Python and serve from a low-latency C++ runtime across TPUs, GPUs and CPUs.
  • TPU Sync, now open-sourced, is a model-state layer that turns chip HBM, host DRAM, shared storage and the network into one memory hierarchy: tiering KV caches off-chip in milliseconds, streaming context across disaggregated prefill and decode nodes, and synchronizing RL weights. Google claims it cuts KV transfer latency and TTFT by more than 90%.
  • Agent Compute Engine automatically suspends and resumes idle agent sessions, attacking the cost of agents that sit paused waiting on tools, databases or humans.
  • Cloud Pathways is now generally available on GKE, bringing the DeepMind runtime behind Gemini to customers for orchestrating multi-slice TPU systems. GKE dynamic slicing is also GA, using optical circuit switching to assemble or partition TPU topologies on demand; Google claims up to 5x faster job startup and 4.5x faster fault recovery.
  • On the industry-standard InferenceX and AgentX benchmarks, Google says its vLLM-TorchTPU stack on TPUs achieved state-of-the-art price-performance. These are vendor-reported results; independent numbers will tell.

What I would check

My untested suggestions before planning around this:

  • Confirm availability and pricing for 8tx and 8i Max with Google Cloud directly; the announcement has neither, and "announced" is not "orderable."
  • If your agents stall on million-token contexts, profile whether you are memory-bandwidth-bound before assuming 8i Max is the fix; the same post ships TPU Sync, which attacks the bottleneck in software on existing hardware.
  • For confidential computing, compare 8tx against GPU-based confidential-computing options on your threat model and latency budget; Google claims up to 2x lower latency than 8t, not versus the field.
  • Test the TorchTPU path on a real model before committing; "no code rewrites" claims deserve a benchmark on your own PyTorch code.
  • If your agents idle waiting on tools, look at Agent Compute Engine's suspend-resume before buying more capacity.

What I would not conclude

Do not conclude these systems are generally available: the post announces them and points to a launch broadcast, but lists no dates or prices. Do not treat the InferenceX and AgentX results as independent; they are Google's own runs of industry benchmarks. Do not read "near-zero performance penalty" for confidential computing as zero; get the number for your workload. And do not assume the 8th-generation portfolio replaces GPUs in Google's own stack; the XLA Serving Engine explicitly serves across TPUs, GPUs and CPUs.

Frequently asked questions

What are TPU 8tx and 8i Max?

Two new eighth-generation TPU systems Google announced on November 9. TPU 8tx is built for confidential inference with hardware-level protection of data and model weights in use. TPU 8i Max combines two TPU 8i chips on one package for double the compute, SRAM and HBM bandwidth, targeting massive context windows.

How many TPU systems are in the eighth generation?

Four, all announced within a single year: 8t for large-scale training, 8i for real-time reasoning, and now 8tx for confidential computing and 8i Max for high-context workloads.

What is confidential inference?

Normally data is encrypted at rest and in transit but exposed in memory during processing. TPU 8tx builds the protection into the hardware layer so sensitive data and model weights stay isolated even while the model is actively using them, aimed at regulated enterprises and governments.

Can PyTorch code run on TPUs now?

Google open-sourced TorchTPU last month and says PyTorch code and Hugging Face models now run natively on TPUs without rewrites, with TPU backends for vLLM and SGLang. As with any portability claim, verify on your own code.

What is TPU Sync?

An open-sourced model-state layer that unifies chip HBM, host DRAM, shared storage and the network into one memory hierarchy for TPU clusters. Google says it tiers KV caches off-chip in milliseconds and cuts KV transfer latency and time to first token by more than 90%.

When can I buy TPU 8tx or 8i Max, and what do they cost?

The announcement includes no availability dates or pricing for the two new systems. That is the main open question to take to Google Cloud.

Are the performance claims independent?

No. The latency, price-performance and benchmark figures in the announcement are Google's own, including its runs of the InferenceX and AgentX industry benchmarks. Independent evaluations have not appeared yet.

Sources

  • Introducing TPU 8tx and TPU 8i Max | Google Blog - November 9, 2026, read November 11, 2026. The announcement, system descriptions, latency and performance claims, software stack (TorchTPU, TPU agent suite, XLA Serving Engine, TPU Sync, Agent Compute Engine, Cloud Pathways, dynamic slicing), and customer count.
  • TLDR AI newsletter, November 10, 2026 - the tip that surfaced the announcement.

Not read: the TPU Sync and TorchTPU repositories, the InferenceX and AgentX benchmark submissions, and the launch broadcast. Pricing and availability for the new systems are unannounced; all performance figures are Google's own.

Related reading

The build log

Follow the work.

New essays and prototypes, in your feed reader. Email subscriptions are paused while I connect a working delivery service. No email is collected here.

Follow via RSS