←
Guanlan Dai448 posts
Guanlan Dai avatar

Guanlan Dai

@guanlan

Building @Runta, the execution layer that controls what AI agents can actually do. Prev @Cloudflare and @Kong.

◷ Joined January 2010▧ 101 Photos and videos
Guanlan Dai avatar
Guanlan Dai@guanlan·Jul 16

Today we're announcing @runta's $20M seed, led by @a16z. Software is constrained when you write it. Agents have to be constrained while they run. Runta is the execution layer that controls what AI agents can actually do.

1:23
Guanlan Dai avatar
Guanlan Dai@guanlan·23h

What we’d do with this: Switching models? Re-test your harness. Its cost rank moved a median of 11 places out of 22. Choose on cost. On the same model, pass rates were mostly close while cost per pass differed by up to 4.4x. Building a harness? Cut steps if possible. Prompt size, tool count and cache hit rate didn't predict cost per pass. Every step sends the whole context again. Full write-up: runta.com/blog/introducing-f…

Guanlan Dai avatar
Guanlan Dai@guanlan·23h

Overall, Codex CLI has the lowest cost per pass ($0.937) and ZCode the highest pass rate (61.9%, with Codex CLI at 61.1%). Where does the money go? Mostly to failed runs. 64% of inference spend went to runs that didn’t pass. Among the best-value harnesses, 68% to 82% of failed runs didn’t time out. They ended with the agent confidently saying it was done.

Post attachmentPost attachment
Guanlan Dai avatar
Guanlan Dai@guanlan·23h

The surprise 22nd harness was OpenCode v2 (2.2.7), as some of you guessed. Against OpenCode v1 (1.19.4) across all five models: same pass rate (57.3% vs 57.4%), about 13% lower cost per pass ($0.978 vs $1.12). Worth the upgrade if you’re still on v1.

Post attachment
Guanlan Dai avatar
Guanlan Dai@guanlan·23h

A small harness doesn’t mean low-cost. mini-SWE-agent has the smallest starting prompt of all, 882 tokens, and ranks 18th of 22 on cost. Across harnesses, prompt size tells you nothing about cost (correlation: +0.01). mini-SWE-agent goes on to use 3.8M input tokens per trial. Pi uses 2.5M.

Post attachment
Guanlan Dai avatar
Guanlan Dai@guanlan·23h

What drives the bill? The model sets most of the bill, explaining 93% of cost variation across the 110 combinations. The harness is the second lever, worth up to 4.4x once the model is fixed.

Post attachment
Guanlan Dai avatar
Guanlan Dai@guanlan·23h

Paying more doesn’t buy a higher pass rate. The harness with the highest pass rate ranked anywhere from 4th to 19th on cost, depending on the model. On four of the five models, the cheapest harness was within 0.4 points of the most expensive on pass rate.

Post attachment
Guanlan Dai avatar
Guanlan Dai@guanlan·23h

FrontierHarness v1.1 results: when you switch models, consider switching harnesses too, at least if you care about your token bill. The full run: 83B tokens across 27,280 agent runs, each in its own @runta runtime, inference by @baseten. Codex CLI ranks 1st on cost with GLM-5.3, and 10th of 22 with Kimi-K3. On any one model, cost per pass varies by up to 4.4x across harnesses. Only Pi never left the top 6 on cost. 🧵

Post attachment
Runta avatar
⟳ Guanlan Dai retweeted
Runta@runta·Nov 15

Cursor/Notion recent incidents showed how stored agent traces can leak credentials. Agent traces and logs can leak. If an agent can read a API key, that key can leak too. At Runta, credentials never enter the runtime. The VM only gets a stub. All outbound traffic goes through our egress gateway, which adds the real key only for the hosts and paths you allow.

Post attachment
Guanlan Dai avatar
Guanlan Dai@guanlan·Nov 13

There’s a surprise 22nd harness in the FrontierHarness v1.1 eval, running alongside the other 21 under the same conditions. We’ll reveal it with the final results. Hint: it’s racing a version of itself. Any guesses?

Post attachment
Guanlan Dai avatar
Guanlan Dai@guanlan·Nov 12

Halfway through FrontierHarness v1.1, the leanest harness costs the least per pass on most models. Pi uses the fewest tokens per trial on all 5 models, 550k to 900k, while others run up to 6M. It ranks #1 on cost per pass on 4 of the 5. Pi ships 4 tools (read, bash, edit, write) and a ~650-token system prompt. Pi + DeepSeek-V4.1-Flash has the highest pass rate so far, close to 80%, at $0.06 a pass. Overall, Codex still leads on cost per pass, carried by Kimi K3. Is lean winning because it’s better designed, or because today’s models no longer need thick scaffolding?

Post attachmentPost attachment
Guanlan Dai avatar
Guanlan Dai@guanlan·Nov 10

Models: GLM-5.3, GLM-5.3-Flash, DeepSeek-V4.1-Flash-1017, Kimi-K3, Muse Spark Open. Inference for the v1.1 run is sponsored by @basetenco and @Zai_org. GLM-5.3 and GLM-5.3-Flash are among the models under test, so to be explicit: @Zai_org has no say in tasks, models or protocol, and they see results when you do.