Google's AI Chips Break Into Meta's Data Centers, Challenging Nvidia's Grip

The seventh-generation Ironwood TPU has just demonstrated a throughput of over 700 tokens per second, a figure that signals a significant shift in the artificial intelligence infrastructure market. This speed was achieved while serving the massive 2.8-trillion-parameter Kimi K3 model, marking a pivotal moment for Google’s custom silicon. For the first time, these chips are running production workloads outside of Google’s own data centers, specifically within the infrastructure of Meta.

This expansion is powered by a new, PyTorch-native version of the SGLang inference engine. Previously, running these specific chips required using Google’s proprietary JAX framework. The new software stack allows Meta to utilize its preferred PyTorch ecosystem without extensive code rewrites. Zongwei Zhou, Meta’s PyTorch-TPU lead, confirmed that the system is already live in production, describing it as a remarkably mature serving stack developed through a tight collaboration between Meta, Google, and the SGLang team.

Meta’s engineering leadership views this integration as a critical step toward hardware independence. Jana van Greunen, director of PyTorch engineering at Meta, stated that the company cannot be constrained by a single hardware architecture. The new TorchTPU implementation, co-developed by Google and Meta’s PyTorch team, enables the direct deployment of PyTorch workloads onto Google’s hardware. This marks a significant evolution from the older PyTorch/XLA bridge, which Google now terms a predecessor.

The strategic timing of this announcement is notable, arriving just one day after Amazon revealed it was open-sourcing its own native PyTorch backend for its Trainium chips. Both moves appear designed to loosen the software lock-in that has long favored Nvidia’s ecosystem. While the exact location of Meta’s production TPUs is not public, reports from last year indicated that negotiations involved Google Cloud rentals for 2026 and potential purchases in 2027, with a multi-year agreement reportedly signed.

Performance benchmarks further underscore the competitiveness of this new hardware-software pairing. Inferact, a startup founded by the creators of vLLM, reported that a tuned setup using 16 Ironwood chips with speculative decoding outperformed Nvidia’s GB200 chips, which recorded 452 tokens per second in comparable tests. At small batch sizes, the new kernels for specific models are delivering roughly 1.4 to 2 times the throughput of the Nvidia baseline, even without the speed-boosting speculative decoding method.

The rollout is not limited to SGLang. Google has also announced an open-source version of vLLM, another widely used inference engine, that supports these new chips natively. This dual-engine approach provides Meta with robust options for serving large language models. The integration represents a major departure from the status quo, where Nvidia’s CUDA ecosystem dominated high-performance AI inference.

Looking ahead, the SGLang-TorchTPU project plans to expand its model support, starting with DeepSeek V4.1 Flash and Reflection Beam. The teams are aiming for Day 0 support for new open models by the first quarter of 2027, ensuring that new releases are immediately compatible. Google, Meta, and RadixArk, the startup behind SGLang, are scheduled to share further details at the SGLang Summit in San Francisco in November.

This development suggests a more fragmented and competitive future for AI hardware. By making its chips accessible to major external players through familiar software frameworks, Google is actively challenging the dominance of the incumbent market leader. For the broader tech industry, the successful integration of these chips into Meta’s production environment serves as proof of concept for a multi-vendor hardware strategy.