Introducing Together Compiler: agents that continually optimize kernels for your model, your hardware, and your traffic.
We built this to scale frontier research from the teams behind FlashAttention, ThunderKittens, ThunderAgent, and more to every customer deployment.
So far we're seeing:
👉 +12% to +20% median TPS on serverless endpoints our researchers had already heavily optimized by hand
👉 Customers in production cutting response and generation times by about a third to nearly half
Read the full story: togetherai.link/R7BKfl7
deepseek v4.1 flash 1017 is now available for post-training on together ai
run sft or rl with your own data, rewards + environments, and keep the training loop in your own python code
take v4.1 flash and shape it for your own coding and agentic work
together.ai/post-training
For languages off-the-shelf speech models underserve, @wonderful_ai, the AI OS for the enterprise, builds its own voice models.
Running on @NVIDIA accelerated computing, through Together AI:
→ 274ms time to first sentence
→ 100% uptime
→ Zero human interaction to scale, ship new weights, or run blue/green tests
together.ai/customers/wonder…
Here's how we do agent evals at Together AI:
→ Define ~10 representative tasks
→ Run agents (Claude Code) in sandboxes
→ Test 3 configs (baseline, MCP, skills) for each task
→ Analyze trajectories and improve docs + APIs
Agent task completion improved from 67% to 96%!
Hassan @nutlope Nov 16 Article How we do agent evals at Together TLDR: I go over how we made Together easier for agents to use, improving agent task success rates from 67% to 96% by building our own agent evals. We'll walk through how we built the evals & how you
coding agents resend the whole conversation on every turn. prefix caching keeps long sessions fast, but only if the harness and the inference engine both do their part.
@zainhas from Together and @parthsareen from @ollama wrote down what each side needs to get right.
together.ai/blog/making-open…
Last Year We Built Megakernels. This Year We Deleted Them.
One year ago, we helped introduce some of the first AI megakernels to the world. Since then, we’ve enjoyed seeing their wide adoption from friends and researchers. However, while whole model megakernels are a helpful abstraction, we’ve found that they overcomplicate and are not always an ideal optimization target.
Below, we discuss our journey to understand and push past these limitations. We start with a deep dive into two contrasting paradigms of ML optimization – Torch Compile and Megakernels
Two weeks ago we announced Together Compiler: agents that tune inference to your model, chip, and traffic, for up to 2.6x more throughput.
Today, we’re sharing the journey behind it: the experiments that challenged our assumptions of megakernels, the surprising strengths of Torch Compile, and the lessons that shaped our development of a next-generation system for inference optimization.
Shout out to @dylan__lim@reyna_abhyankar@simran_s_arora and the team!
Read more: together.ai/blog/road-to-tog…
Two weeks ago we announced Together Compiler: agents that tune inference to your model, chip, and traffic, for up to 2.6x more throughput.
Today, we’re sharing the journey behind it: the experiments that challenged our assumptions of megakernels, the surprising strengths of Torch Compile, and the lessons that shaped our development of a next-generation system for inference optimization.
Shout out to @dylan__lim@reyna_abhyankar@simran_s_arora and the team!
Read more: together.ai/blog/road-to-tog…
in a world where agent experience (AX) >> developer experience (DX) you need to make sure agents can use your platform
this is how we use evals to make sure agents can use the Together platform🔥
Hassan @nutlope Nov 16 Article How we do agent evals at Together TLDR: I go over how we made Together easier for agents to use, improving agent task success rates from 67% to 96% by building our own agent evals. We'll walk through how we built the evals & how you
How we do agent evals at Together:
→ Define ~10 representative tasks
→ Run agents (Claude Code) in sandboxes
→ Test 3 configs (baseline, MCP, skills) for each task
→ Analyze trajectories and improve docs + APIs
Agent task completion improved from 67% to 96%! More below:
closed → open migration works best as an iterative rollout
discover → evaluate → adapt → decide → production
evaluate on real traffic, improve what falls short, then canary the strongest candidate before moving more traffic