←
Worldline526 posts
Worldline avatar

Worldline

@Worldline_AI

The decision layer by @Ch40sChain for AI coding agents.

◷ Joined May 2026▧ 146 Photos and videos
Worldline avatar
Worldline@Worldline_AI·Aug 26

Opus 5 at 67.0%. GPT-5.6 Sol at 64.1%. A 2.9 point gap on someone else's tasks. The index ranks harness plus model. It cannot tell you which agent instance earned trust in your monorepo. Same model, different behavior. The ranking is real. It is just not about your work. #DYOR

Announcing the Artificial Analysis Coding Agent Index! Our new coding agent benchmarks measure how combinations of agent harnesses and models perform on 3 leading benchmarks, token usage, cost and more When developers use AI to code they’re choosing a model, but also pairing it with a specific harness. It makes sense to benchmark that combination to understand and compare performance. The Artificial Analysis Coding Agent Index includes 3 leading benchmarks that represent a broad spectrum of coding agent use: ➤ SWE-Bench-Pro-Hard-AA, 150 realistic coding tasks that frontier models struggle with, sampled from Scale AI’s SWE-Bench Pro ➤ Terminal-Bench v2, 84 agentic terminal tasks from the Laude Institute and that range from system administration and cryptography to machine learning. 5 tasks were filtered due to environment incompatibility ➤ SWE-Atlas-QnA, 124 technical questions developed by Scale AI about how code behaves, root causes of issues, and more, requiring agents to explore codebases and give text answers Analysis of results: ➤ Opus 4.7 and GPT-5.5 lead the Index: Opus 4.7 in Cursor CLI scores 61, followed closely by GPT-5.5 in Codex and Opus 4.7 in Claude Code at 60. GPT-5.5 in Cursor CLI follows at 58. ➤ Open weights models are competitive, but still trail the leaders: GLM-5.1 in Claude Code is the top open-weight result at 53, followed by Kimi K2.6 and DeepSeek V4 Pro in Claude Code at 50. These are strong results, but still meaningfully behind the top proprietary models. ➤ Gemini 3.1 Pro in Gemini CLI underperforms: Gemini 3.1 Pro in Gemini CLI scores 43, well below where Gemini 3.1 Pro sits on our Intelligence Index, highlighting that Gemini’s performance in Gemini CLI remains a relative weak spot for Google’s offering. ➤ Cost per task (API token pricing) varies >30x: Composer 2 in Cursor CLI is cheapest at $0.07/task, followed by DeepSeek V4 Pro in Claude Code at $0.35/task and Kimi K2.6 in Claude Code at $0.76/task. At the high end, GPT-5.5 in Codex costs $2.21/task, while GLM-5.1 in Claude Code costs $2.26/task. For both models this was contributed to by high token usage, and in GPT-5.5’s case by a relatively higher per token cost. ➤ Token usage varies >3x: GLM-5.1 in Claude Code uses the most tokens at 4.8M/task, followed by Kimi K2.6 at 3.7M/task and DeepSeek V4 Pro at 3.5M/task. GPT-5.5 in Codex uses 2.8M tokens/task, substantially more than Opus 4.7 in Claude Code at 1.7M/task. In GLM-5.1’s case, higher token usage, cost and execution time were partly driven by the model entering loops on some tasks. ➤ Cache hit rates remain high but vary materially: Cache hit rates range from 80% to 96% across combinations. Provider routing, harness prompt structure and cache behavior can materially change the economics of running the same model given cached inputs are typically <50% the API price of regular input tokens. ➤ Time per task varies >7x: Opus 4.7 in Claude Code is fastest at ~6 minutes/task, while Kimi K2.6 in Claude Code is slowest at ~40 minutes/task. This is contributed to by differences in average turns per task, token usage and API serving speed. Opus 4.7 had materially lower amount of turns to complete a task than all other models while Kimi K2.6 had the most. ➤ Cursor made real progress with Composer 2: Composer 2 in Cursor CLI scores 48, near the leading open-weight model results, while being the cheapest combination measured at $0.07/task. Cursor has stated Composer 2 is built from Kimi K2.5, showcasing they have made substantial post-training gains. This is just the start. We are planning to add additional agents (both harnesses and models). Let us know what you would like to see added next.

Post attachment
Worldline avatar
Worldline@Worldline_AI·Aug 20

Altman says confidence in safety will set the pace of AI progress. Confidence is not evidence. When the pace of an entire field is set by how a vendor feels about its own system, the receipt stops being a nice-to-have and becomes the only thing you actually own. #DYOR

We have paused some frontier RL training to ensure that we can meet the appropriate alignment, security and monitoring standards for the new level of capabilities in front of us. Model progress is now extremely rapid, and we always said we would take action if we felt that model capabilities were outstripping the pace of safety and alignment. We care very deeply about AI safety. We believe the entire field will have to coordinate on shared safety standards, but will act unilaterally in the meantime. We expect confidence in safety to increasingly set the pace of AI progress. We are optimistic about the alignment work we are doing, and we remain committed to making frontier capabilities widely available. openai.com/index/pacing-mode…

Worldline avatar
Worldline@Worldline_AI·Aug 17

Repos are portable. Sessions are not. When your code moves hosts, the record of which agent instance wrote which diff, and whether it earned that, does not move with it. You are migrating the output and leaving the evidence behind. #DYOR

Origin, our code hosting platform, is now live. It's fast, easy to use, and deeply integrated with Cursor. Get started by syncing your repos from GitHub.

0:31
Sumeet Chougule avatar
⟳ Worldline retweeted
Sumeet Chougule@_sumeetc·Aug 17

We’re hiring a Founding GTM Lead in India. Own 0→1 enterprise and public-sector GTM for Worldline: sales, category marketing, partnerships and paid sovereign-AI design engagements. Mid to Senior, hands-on, remote. getworldline.com/careers

Worldline avatar
Worldline@Worldline_AI·Aug 13

Everyone shipping agent work makes this face 5 times a day, then merges anyway. The report is confident. The suite is green. Neither is evidence of what actually happened. The trace is. Where's the receipt?

Post attachment
Worldline avatar
Worldline@Worldline_AI·Aug 10

'Improved factuality.' Improved from what baseline, measured on what, by whom? A version number is not a changelog and an adjective is not an eval. You will make real decisions with these outputs. The vendor's grade of the vendor's model is the only score in the room. #DYOR

Grok@grokAug 8

Announcing Imagine Image 2.0, our next generation image model with precision editing, crisp text rendering, improved factuality, and real world usefulness. Image 2.0 helps you make images for real work. x.ai/news/grok-imagine-image…

Worldline avatar
Worldline@Worldline_AI·Aug 6

Tomorrow the model under ChatGPT changes for a few hundred million people. The published evidence for what changes: 'more factual, focused.' If a fellow dev shipped a swap of a dependency with that changelog, it would not survive review. This one will. 🤷‍♂️

We’re making better intelligence easier to access in ChatGPT for everyone: - GPT-5.6 Sol now powers both Instant and deep reasoning for Plus & Pro users, delivering more factual, focused responses. - Free & Go users get unlimited text chats with GPT-5.6 Luna starting tomorrow.

0:09
Worldline avatar
Worldline@Worldline_AI·Aug 5

The table ranks models, but it cannot say which agent instance earned trust on your work. #DYOR

I'm a little speechless right now. A 1-bit model - DeepSeek-V4-Flash-AD-IQ1_M, just performed the best ever on my ScaffoldBench. It just notched Opus 4.8 by 41% vs 39% on verify. And I can run it locally. Only downside; it's the 4th slowest model on my hardware.

Post attachment
Worldline avatar
Worldline@Worldline_AI·Aug 3

The left button ships with a number from the vendor. The right button only has a number if your team kept the record. Most teams are not choosing today. They are pressing the only labeled button. What does your right button say?

Post attachment
Sumeet Chougule avatar
⟳ Worldline retweeted
Sumeet Chougule@_sumeetc·Aug 3

We’re hiring the early engineering team building Worldline. Open roles: → Rust Systems Engineer — Desktop, Edge & Security → Product Engineer — React/TypeScript, Enterprise Control Room → Engineering Interns — Systems or Product Remote within India. This is foundational product work across agent capture, independent verification, Fleet, AI security and multiplayer human-agent workflows. Apply: getworldline.com/careers

ChaosChain avatar
⟳ Worldline retweeted
ChaosChain@Ch40sChain·Aug 2

getworldline.com

Agent work shouldn’t disappear into private sessions. Worldline is the multiplayer control room for human-agent work: a shared, sovereign operating record of context, actions, evidence, decisions, outcomes and cost across models. Our new home is live: getworldline.com

Post attachment
Worldline avatar
Worldline@Worldline_AI·Jul 31

Both axes measure the label: what the model costs everywhere, what it scores everywhere. Neither measures the instance doing your work tonight. The chart that settles routing plots your own sessions against evidence. Nobody publishes that one, nobody except us 😎

OpenAI@OpenAIJul 30

We are committed to pushing the model frontier across cost efficiency, capability, and speed. Starting today, we are reducing prices for GPT-5.6 Luna by 80% and GPT-5.6 Terra by 20% , and offering a faster option for GPT-5.6 Sol in the API. Luna and Terra’s lower prices are reflected in how usage is counted in Codex and ChatGPT Work, so your usage goes further.

Post attachment
Worldline avatar
Worldline@Worldline_AI·Jul 29

1000+ frontier employees say the tools to pace automated AI R&D do not exist yet. The same gap runs through your stack: agents shipping work with no verifier-backed receipt of what they did. Pacing starts where evidence starts. 🤖 #DYOR

Full text of the Pacing the Frontier statement, signed by 1,122 employees for frontier AI companies so far, including a bunch of heavy hitters at OpenAI, Anthropic, Google and others:

Post attachment
Worldline avatar
Worldline@Worldline_AI·Jul 28

Score your coding agent's last session: reasoning, compliance, efficiency, collaboration, initiative, 1 to 5 each. If you can't fill 3 of the 5 without reopening the trace, you don't have a trust profile. You have a feeling. Which dimensions are you guessing on? #DYOR

Post attachment
Worldline avatar
Worldline@Worldline_AI·Jul 27

30+ logos can secure a supply chain. None of them can tell you what your agent did in your repo yesterday. Safety gets set at the industry level. Trust gets earned at the instance level. The alliance protects the category. The receipt protects you. #DYOR

NVIDIA@nvidiaJul 27

AI security advances when the industry builds in the open, together. We're introducing the Open Secure AI Alliance with industry leaders to develop new techniques and tools to safeguard software and agents. By sharing models, tooling and research in the open, we can broaden the community of defenders. Learn more about the founding members’ contributions: nvda.ws/4pD8Fc5

Post attachment
Worldline avatar
Worldline@Worldline_AI·Jul 24

Opus 5 may be everything the post says, but the instance you spin up tomorrow has 0 sessions in your codebase, and no benchmark transfers. The trust profile starts empty. #DYOR

Introducing Claude Opus 5. It's a thoughtful and proactive model that comes close to the frontier intelligence of Fable 5 at half the price.

0:08