TorchTPU Is Open Source: What Google and Meta Say Running Native PyTorch on TPUs Looks Like

Google open-sourced TorchTPU on October 20, at PyTorch Conference 2026. It is a PyTorch backend for Google TPUs, built with the PyTorch team at Meta, that aims to let you move a model from GPU to TPU by changing the device name. The coverage and speed figures come from Google and the PyTorch Foundation's own posts, and I have not run the code.

I read two primary sources: Google Cloud's TorchTPU is now open source and the PyTorch blog's Introducing TorchTPU. I read the PyTorch post through its architecture and results sections, but not every part. I did not open the GitHub repository or the linked arXiv paper.

What it is

Per the PyTorch post, TorchTPU started at Google in mid 2025 with the PyTorch team at Meta. It replaces the older approach, PyTorch/XLA from 2020, which the post says needed intrusive code changes, had severe compile-time latency, lacked eager execution and lagged behind new PyTorch features.

The stated principles: no JAX dependency in the runtime, solid eager mode, torch.compile built on Dynamo, support for PyTorch distributed tools such as DDP, FSDP and DTensor, and a porting step as simple as changing model.to("cuda") to model.to("tpu"). The post adds that this will run a model but will not reach peak TPU performance, because shapes, sharding and ops may favour GPUs.

What the PyTorch post says it supports today

• Eager and compiled modes, including a new mode called DeferAndFuse that defers and fuses ops into larger chunks. The post says it can be 50% or more faster than strict eager and use less peak memory.
• About 450 Aten ops, with forward and backward for the core ops needed by major architectures.
• Custom kernels through JAX/Pallas and the Helion DSL.
• DTensor sharding, multi-host inference with vLLM and SGLang, and training with TorchTitan, orchestrated with Ray.
• Compatibility with Hugging Face Transformers v5, Diffusers and Kernels.

The post says it is used in production for inference and training by some of the largest Google Cloud customers. It names no customers, so I cannot check that.

Coverage numbers

The post gives these results at the time of writing:

Measure	Result
Top-downloaded models, runs without a crash	98%, 4,821 of 4,917
Representative architectures, numerical parity with GPU	97%, 638 of 657
Bounded dynamism, correct numerics on causal LLMs	99%, 834 of 839
Single TPU v7 chip prefill, sequence length 5,120	about 25.7k tokens per second, 59% MFU
Single TPU v7 chip prefill, sequence length 10,240	about 22.4k tokens per second, 51% MFU

"Runs without a crash" is not "runs fast", and parity is measured against GPU output. These are the authors' own tests on models they chose from a download ranking.

Limits the post states

This part is worth reading closely, because it is candid:

• A TPU can be accessed by only one user-space process, so RL colocation does not work out of the box. The authors say they are investigating workarounds.
• XLA assumes tensor shapes are known at compile time. TorchTPU adds bounded dynamism, where only maximum shapes must be known, to reduce recompilation.
• By default each collective op compiles in its own module, which the post says can cost 30 to 40% end-to-end compared with the faster option that lets the compiler overlap compute and communication.
• Reaching peak performance still needs shapes wide enough to fill the TPU's matrix units.

The Google Cloud announcements around it

The Google Cloud post adds integrations. Treat all of it as Google's description:

• Hugging Face. In Transformers v5, deploying to TPU is device_map="tpu", with nightly TPU testing of the Transformers suite.
• vLLM and SGLang. Native PyTorch backends, built with Inferact and RadixArk, that Google says keep upstream schedulers and API servers.
• llm-d 1.0. Released with Red Hat, IBM and a community Google puts at more than 500 contributors and 25 organizations, as a CNCF Sandbox project with prefix-cache-aware routing. A customer quote claims over 4 times higher aggregate throughput on GKE; it is the customer's claim.
• TorchTitan and VeRL. TorchTitan recipes validated on up to 8,000 Ironwood TPUs, and a VeRL TPU backend marked as preview.
• Ray, LiteRT and a TPU agent suite. Ray orchestration across CPUs, GPUs and TPUs, LiteRT joining the PyTorch ecosystem, and an early-preview agent suite on GitHub for onboarding and tuning.

What I would check

These are my own untested suggestions.

1. Run your own model. The 98% figure is across popular models. Yours may use custom ops.

2. Measure speed, not just that it runs. Compare cost per token or step against your GPU baseline after tuning shapes.

3. Check dynamic shapes. If your batch or sequence length varies, test recompilation behaviour.

4. Look at the collectives setting if you train across chips, given the 30 to 40% figure.

5. Check RL needs. If you rely on colocating trainer and sampler on one device, the single-process limit matters.

6. Check availability and price of the TPU generations you need on Google Cloud. I read nothing on this.

What I would not conclude

• That TPUs now match GPUs for every PyTorch workload. The authors say shapes and ops still matter.
• That the production claim is independently confirmed. No customers are named.
• That a one-line port gives good performance. The post says it only gives a working model.
• That every integration is production ready. VeRL is labelled preview, and the agent suite is early preview.

Frequently asked questions

What is TorchTPU?

A native PyTorch backend for Google TPUs, developed by Google with Meta's PyTorch team and released as open source.

How is it different from PyTorch/XLA?

Per the PyTorch post, it supports eager mode, uses Dynamo-based torch.compile, and avoids the intrusive code changes and long compile times of PyTorch/XLA.

How do I run a model on a TPU?

The post says porting should be as simple as using model.to("tpu"), though peak performance needs tuning.

Does it work with Hugging Face models?

Google says Transformers v5 models deploy with device_map="tpu", and the PyTorch post reports out-of-box execution on the 5,000 most downloaded Hub models.

Does it work with vLLM and SGLang?

Yes, according to both posts, through native PyTorch backends.

What are the main limits?

Single process per TPU device, static-shape assumptions handled with bounded dynamism, and a default collectives mode with a reported 30 to 40% cost.

Is it ready for production?

The PyTorch post says it is used in production by some large Google Cloud customers, without naming them. Test your own workload.

Sources

• Google Cloud Blog, TorchTPU is now open source: Run native PyTorch on TPUs (October 20, 2026): integrations, partner quotes and availability notes.
• PyTorch Foundation, Introducing TorchTPU (October 21, 2026): principles, supported features, coverage results, limits. Read through the results section.