←
BackboardIO1,562 posts
BackboardIO avatar

BackboardIO

@BackboardIO

a new way to: inference. pre-training. post-training. quant. kerneling where no one has kerneled before.

◷ Joined October 2025
BackboardIO avatar
BackboardIO@BackboardIO·Nov 16

Agentic AI needs performance, not just the right model. We tested Backboard on Qwen3.8-Flash-Next with the AgentX benchmark, and the results made one thing very clear: Inference performance matters.

BackboardIO avatar
BackboardIO@BackboardIO·Nov 16

Week one done. What’s next? More models, newer hardware, matched-precision runs, and an open invitation that doesn't expire: bring us your workload, or run our engine on your benchmark, and we'll publish what comes out. Intelligence should be cheap enough to be reckless. We're working on it.

BackboardIO avatar
BackboardIO@BackboardIO·Nov 15

What an inference engine actually does, in plain terms: A model generates one token at a time. The engine decides which requests share the GPU, what stays in memory (the KV cache), and in what order work runs. Those scheduling and memory decisions are worth more than most people think. This week: 12.8x worth, in the best case we’ve measured. backboard.io/blog/qwen-flash…

BackboardIO avatar
BackboardIO@BackboardIO·Nov 15

One week of posting our AgentX numbers in public. The questions we got were better than the compliments. More benchmarks should be run this way: in the open, with the conditions on the page.

BackboardIO avatar
BackboardIO@BackboardIO·Nov 15

Make intelligence cheap enough to be reckless. That's the whole of Backboard in one line. When inference is cheap enough, people stop rationing it and start experimenting with it. Everything we build is in service of that price coming down.

BackboardIO avatar
BackboardIO@BackboardIO·Nov 14

If you could have one inference benchmark run properly with the same hardware, same conditions, and full tables published, what would it be? Tell us about the model, workload, and metrics. We'll take the best answer and run it.

BackboardIO avatar
BackboardIO@BackboardIO·Nov 14

It’s been days since we pitted our AgentX numbers against SGLang's published results and offered our engine for an independent run at @SemiAnalysis_ The offer remains: same benchmark, same workload, run by whoever wants to check our math. We'll keep publishing ours either way. backboard.io/blog/qwen-flash…

BackboardIO avatar
BackboardIO@BackboardIO·Nov 13

We have a standing offer: @SemiAnalysis_, run our engine on AgentX yourselves. Same containers, same workload, your boxes. We'll publish whatever you find. That's the comparison we want.

BackboardIO avatar
BackboardIO@BackboardIO·Nov 13

What we're testing next, in the open: More models. Newer hardware generations. Matched-precision comparisons, BF16 against BF16, so the engine is the only variable. If there's a workload you think breaks us, tell us. That's the most useful thing you can send.

BackboardIO avatar
BackboardIO@BackboardIO·Nov 13

A question for the people serving open models in production: Where does your stack actually hurt: time to first token, interactivity under load, or raw throughput per GPU? We build an inference engine, so your answers are our roadmap. @spheron @friendliai, you've both published on this. What are you seeing? ow.ly/46zz51062kV

BackboardIO avatar
BackboardIO@BackboardIO·Nov 12

AgentX sessions aren’t short prompts. They reflect heavy workloads. In Backboard’s run, the average session carried 90,000 to 100,000 input tokens of shared context, across multi-turn conversations with KV-cache reuse between turns. Engines that appear fine with 1,000-token prompts can fall over. This is the regime that matters now. ow.ly/5RRs5106ezl

Post attachment
BackboardIO avatar
BackboardIO@BackboardIO·Nov 12

We tested Gemma 4 12B on a single RTX PRO 6000 Blackwell GPU. Backboard beat both vLLM and SGLang across all 104 measured comparisons. More performance from the same hardware.

BackboardIO avatar
BackboardIO@BackboardIO·Nov 12

There are three kinds of benchmark claims, and they simply aren't the same. Internal: we ran it. Independently evaluated: someone else ran it. Official leaderboard: the benchmark owner ran it. We'll label everything so you always know which you're looking at. Transparency matters.

BackboardIO avatar
BackboardIO@BackboardIO·Nov 12

You can gauge the strength of an inference system without taking risks. We license our engine to cloud providers, inference platforms, and enterprises. It runs on your GPUs, your models, and your workload. Bring us the workload you serve today. We'll measure against your current stack, using the same rules we publish by: constant conditions, full result tables, and included failures. ow.ly/AzMW5105cVn

Post attachment
BackboardIO avatar
BackboardIO@BackboardIO·Nov 11

Output throughput is the work that counts with AI inference: it’s the tokens you actually generate with each GPU. Backboard leads at every concurrency level with Qwen in SemiAnalysis’ AgentX benchmark, with gains ranging from 1.1x to 2.2x. Responsiveness and capacity don't have to involve tradeoffs. You can get both with the right engine. ow.ly/YLeE5105cUE

Post attachment