ALERT 🚨: Trainium3 Neuron profiling & trace stack is so fine-grained and precise that it can generate pixel art of entire videos!
The video below is an actual profile running on Trainium3 DMA engines.
We will open-source the entire SKILLS.md later this month.
Great work by @radixark, @sgl_project, @BanghuaZ, @alxnails, and AWS on this amazing work on Trainium! Excited to get SGLang Trainium onto the industry-standard AgentX/InferenceX!
🚀 SGLang-Neuron v0.1.0 is now open source and in public preview!
Following October's closed beta, SGLang's serving stack now runs on AWS Trainium, co-developed by the SGLang team and AWS Annapurna Labs @AmazonScience.
A PyTorch-native plugin on upstream SGLang for Trainium2 and Trainium3:
- One fully asynchronous runtime for every LLM: Trainium chips run decode steps back to back
- Megakernels: every decoder layer of a step in a single NKI kernel call
- PD disaggregation over EFA: Kimi-K3 across three Trainium3 nodes, GLM-5.2 at 1M context
- HiCache: evicted prefixes stay in host memory and load back on a later hit
- SGLang Diffusion on Trainium2: FLUX.1-dev, FLUX.3-Action, MiniMax-H3, Wan2.2 A14B, LingBot-World-V2
Thank you to the AWS Annapurna Labs team and @YottaLabs for the contributions!
Congratulations to the Amazon Last Mile team on winning the 2026 @INFORMS Daniel H. Wagner Prize. Their paper details a planning system that accounts for how temperature and terrain change a delivery EV's energy needs. amzn.to/4iHcbQU ⚡
The SGLang workflow you already know now runs on AWS Trainium.
SGLang-Neuron: serve models and generate RL rollouts on Trainium — same server, same APIs, native PyTorch through TorchNeuron. @sgl_project#SGLangSummitaws.amazon.com/blogs/machine…
Introducing Mitra-v2, a tabular foundation model for both classification and regression. Pretrained entirely on synthetic tables, it leads the single-model field on the TabArena benchmark. amazon.science/blog/mitra-v2…
📣 Last call: Amazon Research Awards Fall 2026 proposals are due November 11. Successful applicants receive unrestricted funds, AWS promotional credits, and training resources. amazon.science/research-awar…
How do you keep 70-billion-parameter quality while running up to 20 times as fast? AWS researchers combined knowledge distillation with parallel decoding. For @bookingcom's Smart Filters, that meant responses 15 times as fast for 99% of queries. amazon.science/blog/speeding…
A text-to-speech system that knows which character is speaking, and voices them accordingly. @audible_com's controllable TTS profiles characters and attributes dialogue with 97% accuracy in English: amazon.science/blog/better-s…
📢 The Amazon Research Awards Fall 2026 deadline has been extended to November 11.
Award recipients receive unrestricted funds, AWS promotional credits, and training resources. Submit your proposal today: amazon.science/research-awar…
Amazon's custom Trainium chip helps power the diffusion models behind Prime 360's cinematic highlights, reducing diffusion-model inference latency by more than 40%: amzn.to/4e1puZN
Most AI developers use PyTorch to build their models. And this week @AmazonScience announced a new PyTorch backend for AWS Trainium chips - TorchNeuron. Learn more about the science behind it.
TorchNeuron is now open source. Our new @PyTorch backend translates code into operations executable on AWS Trainium with minimal or no code changes. Learn how the team used AI agents and hundreds of thousands of models to validate it: amazon.science/blog/the-scie…
TorchNeuron is now open source. Our new @PyTorch backend translates code into operations executable on AWS Trainium with minimal or no code changes. Learn how the team used AI agents and hundreds of thousands of models to validate it: amazon.science/blog/the-scie…