Direct answer: Google has open-sourced TorchTPU, its long-awaited version of PyTorch adapted for Google’s TPU chips — letting AI apps move off Nvidia hardware and onto Google Cloud with a simple configuration change and no rewriting. The announcement lands a day after Amazon released TorchNeuron, its equivalent for AWS’s Trainium chips. Together, the two cloud giants are trying to break the default that most AI apps run on Nvidia. Don’t expect a mass exodus — but Nvidia’s lock-in just got its most serious challenge yet.
What happened
PyTorch is the open-source toolkit that AI app developers use to build, train, and run models — by some accounts it powers about 85% of AI apps. It was developed by Meta back in 2016, when Nvidia’s GPUs were already the standard for AI research, so much of PyTorch was built around Nvidia hardware by default. It has been that way ever since.
On Tuesday, Google announced that TorchTPU is now an open-source project. It lets AI apps run on Google’s Tensor Processing Units and — the key selling point — makes porting existing PyTorch apps a matter of configuration, not rewriting. Google says it worked directly with Meta, Hugging Face, and the makers of popular serving tools vLLM (Inferact) and SGLang (RadixArk) so TorchTPU behaves like the PyTorch developers already know. It also launched a TPU agent suite to help developers migrate and fix errors along the way, and updated LiteRT, its tool for running AI on phones, laptops, and embedded devices.
Amazon moved first. On Monday it released TorchNeuron, its PyTorch variant for Trainium chips, with the same no-rewriting pitch — and went a step further by submitting code to the main PyTorch project that decouples features from Nvidia chips, making PyTorch itself more hardware-neutral by default.
What it means for UK small businesses
You don’t buy TPUs. But if you use AI in your business, you rent the computers the AI runs on, and those costs are set by exactly this fight:
- Compute is your hidden AI bill. Every chatbot reply, every AI-generated image, every agent run burns GPU time in a cloud data centre. A genuine three-way contest between Nvidia, Google, and Amazon for AI workloads should, over time, push those prices down — or at least stop them rising.
- Portability is leverage. The reason this matters is lock-in: if your AI setup only runs well on one provider’s hardware, you’re a price-taker. The practical lesson is to keep AI workloads portable — containerised, standard frameworks — so you can actually shop around when prices move.
- Watch the on-device angle. Google’s LiteRT update points at AI running on phones and laptops rather than in the cloud. For customer-facing tools, that means faster responses, no per-query cloud bill, and data that never leaves the device — three things small businesses should care about.
- Infra investors smell margin. This is part of a bigger money story — AI data-centre startups raised billions this year chasing exactly this compute crunch (see our coverage of the $33bn AI data-centre funding wave). When giants fight over the plumbing, the pipes get cheaper.
Honest limits / what to watch
- Nvidia isn’t going anywhere. As TechCrunch puts it, developers won’t ditch Nvidia en masse just because they can. CUDA — Nvidia’s software ecosystem — has a decade of tooling, talent, and habit behind it. A config flag doesn’t erase that.
- This is cloud vendors fighting each other, not a gift to you. Google wants you on Google Cloud; Amazon wants you on AWS. The “open” move is a land grab, and the savings only materialise if providers pass them on.
- Portability promises need testing. “No rewriting” is a claim, not a guarantee — real apps have quirks, custom kernels, and edge cases. Anyone migrating should budget for surprises.
- The real decoupling play is Amazon’s. Google’s move helps Google’s cloud. Amazon’s code submission to PyTorch itself — making the framework hardware-neutral by default — is the one that could genuinely loosen Nvidia’s hold for everyone. Watch whether that contribution lands and sticks.
Bottom line: two of the world’s biggest cloud companies just removed the main technical excuse for staying on Nvidia. The economics of AI compute are about to get more competitive, and any business paying for AI workloads should keep its options — and its architectures — open.
FAQs
What is TorchTPU?
It’s Google’s version of PyTorch — the open-source toolkit used by roughly 85% of AI apps — adapted to run on Google’s Tensor Processing Units (TPUs) instead of Nvidia’s GPUs. Google released it as an open-source project and says existing PyTorch apps can move to TPUs with a simple configuration change, no rewriting needed.
Why is Google doing this now?
Because most AI apps are written by default to run on Nvidia chips, locking developers into Nvidia hardware and, by extension, the clouds that offer it. TorchTPU makes it easy to move those apps to Google Cloud. Amazon made the same play a day earlier with TorchNeuron for its Trainium chips.
What did Amazon announce?
On Monday, Amazon released TorchNeuron, its own PyTorch variant for Trainium chips and AWS — also requiring no rewriting. Amazon went further, submitting code to the main PyTorch project that decouples features from Nvidia hardware, making PyTorch more hardware-neutral by default.
Will developers actually switch from Nvidia?
Not automatically. Google and Amazon have removed the engineering friction, but developers choose hardware on performance, cost, and familiarity. TechCrunch notes nobody will ditch Nvidia en masse just because they can — but the lock-in advantage is now weaker than ever.
How could this affect my business’s AI costs?
AI compute is one of the biggest line items in any AI project. Genuine chip competition should push cloud AI prices down over time — and if you run AI workloads in the cloud, it pays to keep your setup portable so you can shop between providers.
What is Google doing to win developers specifically?
Google says it worked directly with Meta, Hugging Face, the makers of vLLM (Inferact) and SGLang (RadixArk) so TorchTPU works with the tools developers already use, launched a TPU agent suite to help migrate apps and fix errors, and updated its LiteRT tool for running AI on phones, laptops, and embedded devices.
Sources: TechCrunch
