Your last hardware decision shouldn’t decide what you build next.

Today we launched TorchNeuron, which developers can use to run their existing PyTorch code natively on AWS Trainium, the AI chips we design.

Trying a new AI chip used to be a lot of work. You often had to rewrite your code, check that it still gave the right answers, and spend many weeks on it, betting that the new chip was the right one for your workload. That cost time and money, and it wore on teams. Engineers spent that time adapting software instead of improving the product. 

TorchNeuron helps take that off the table. PyTorch is the software most AI teams use to build and run their models. In most cases, teams can try Trainium with a configuration change rather than a code or model rewrite, and popular libraries like Hugging Face Transformers now support Trainium directly. Combined with Trainium’s price-performance, TorchNeuron will make it easier for many more companies to use AI cost-effectively. 

It also changes what teams can choose. The options within a team’s reach shape what it sets out to build. Developers should be able to pick hardware on price, performance, and availability, without the code they already wrote making the decision for them. When moving is expensive, that decision gets made once and then quietly limits everything that follows. When moving is cheap, teams can choose deliberately, project by project. 

We’ve seen this play out many times at AWS. When we launched EC2 in 2006, we offered one type of virtual compute server. Today we offer hundreds, with different processors, memory, and networking. Each time we added variety, customers put it to use, and each new option let some of them build something they couldn’t build well before. We plan to keep going.  

Your first choice shouldn’t set your limits. A new team rarely knows exactly how much computing power it will need or which kind will suit it best, and in AI the models and chips that look best today may not look best next year. If the first choice locks in, it also locks in what the team can build later. On AWS, a team can start with a reasonable choice and change it as its plans grow. With TorchNeuron, a team can keep its model in standard PyTorch and preserve more flexibility to change hardware later, so its first decision doesn’t have to be its final one. 

Choose hardware to suit each application. As teams mature, they use AI in more places, and each place asks for something different. A support assistant needs fast answers, while a system that processes millions of recordings overnight needs a low cost per hour of audio. Teams that can choose what fits each application can design each one around its own needs. 

Deepgram, which turns speech into text for more than 200,000 developers, made exactly this kind of choice. Working with our Generative AI Innovation Center, they moved their speech model to Trainium using the PyTorch tools their engineers already use every day. In the benchmarks they evaluated, Trainium processed about 60% more audio per dollar than their existing GPU setup. Deepgram says that lets them bring better models to customers at lower prices, so a decision about hardware shows up directly in what their customers get. 

Reconsider the ideas you ruled out. Some ideas never make it into a plan because the tools or the economics put them out of reach. Teams learn not to propose them, and after a while nobody remembers the limit was ever there. Widening the options puts those ideas back in scope as deliberate goals. A voice company that once ruled out processing its entire archive of recordings may now have a practical way to do it. A research team can design a model around the approach it believes in, knowing it has more than one place to run it. 

Making choice easy is harder than offering a single path. The easier we make it to use different technologies, the more ideas teams can realistically pursue. TorchNeuron does that for AI chips, giving more teams a practical way to use Trainium. 

TorchNeuron is open source under the Apache 2.0 license and enters public beta today. We worked closely with the PyTorch team at Meta, and contributed our changes to PyTorch itself, making it easier to run PyTorch on other hardware in the future.

We also worked closely with Hugging Face to make Trainium a native device across their libraries, including Transformers, Diffusers, PEFT, and TRL. The same scripts developers run on other hardware now run on Trainium, and 100 most-used text-generation model architectures work out of the box. Models trained on Trainium are saved as standard checkpoints that load on other supported hardware, so the choice stays open in both directions.