←
nilenso1,937 posts
nilenso avatar

nilenso

@nilenso

Employee-owned programmer cooperative in Bangalore.

◷ Joined May 2013▧ 248 Photos and videos
nilenso avatar
nilenso@nilenso·3 Aug 2023

In 2013, a group of makers got together to find new ways to work together. A lot has happened since. We recently celebrated our 10th birthday :) Over the years, we've had the privilege of working with some exceptional organizations and doing work we're proud of. 1/2

Post attachment
Srihari Sriraman avatar
⟳ nilenso retweeted
Srihari Sriraman@SrihariSriraman·Jul 1

Swyx wearing the @tracesdotcom shirt around AIE WF was unexpected and awesome! @swyx you’ve been very kind, and quite helpful too! Thank you for having us present, and thank you for AIE!

Post attachment
Srihari Sriraman avatar
⟳ nilenso retweeted
Srihari Sriraman@SrihariSriraman·Jun 29

I'm at @aiDotEngineer WF with @AtharvaRaykar, and we'll be presenting our work on context-viewer in the Expo. Come over and lets talk about trace intelligence, error analysis, empirical rigor, semantic data extraction and continuous improvement!

Post attachmentPost attachment
Tarun Sachdeva avatar
⟳ nilenso retweeted
Tarun Sachdeva@tarunsachdeva·Jun 19

what if you could have an agent search over your team's session history? - what did we get done this week? - should we skillify things we're all doing? - who is using agents in the most interesting ways? now you can! our new agent is available for our most active users

0:10
Srihari Sriraman avatar
⟳ nilenso retweeted
Srihari Sriraman@SrihariSriraman·Apr 23

I wanted to check this on newer models, but no SWE-bench Pro trajectories exist for Opus 4.6 / GPT-5.4. So I pulled @badlogicgames' issue-fixing trajectories and ran the same analysis. Thanks for putting those out in public, they make this kind of analysis possible. Opus's first edit sits at 47% in your pi sessions, vs 35% for Sonnet 4.5 on SWE-bench. Harness and model differ too, so I can't isolate the prompt's effect, but the shape shifts in the direction you'd expect from the explicit analyze-dont-edit prompt. I think we can see the effect of the human-steering through explicit analysis / go-ahead / wrap-up cues in this comparison.

Post attachmentPost attachment
Mario Zechner avatar
⟳ nilenso retweeted
Mario Zechner@badlogicgames·Apr 23

super interesting work! glad my open traces on @huggingface allow this! my workflow is mostly: - prompt template with injected gh issue url.and instructions on how to annalyze and present results + concise impl plan to me - i confirm analysis/plan either by knowing or double checking manually. may steer to adjust plan a few times until model knows what to do - tell model to implement - check results, steer if necessary - if all good (type checking, linting, tests, manual code review, manual tests), another prompt template is used to wrap up, i.e. changelog, docs, commit, push, comment on issue, close issue

I wanted to check this on newer models, but no SWE-bench Pro trajectories exist for Opus 4.6 / GPT-5.4. So I pulled @badlogicgames' issue-fixing trajectories and ran the same analysis. Thanks for putting those out in public, they make this kind of analysis possible. Opus's first edit sits at 47% in your pi sessions, vs 35% for Sonnet 4.5 on SWE-bench. Harness and model differ too, so I can't isolate the prompt's effect, but the shape shifts in the direction you'd expect from the explicit analyze-dont-edit prompt. I think we can see the effect of the human-steering through explicit analysis / go-ahead / wrap-up cues in this comparison.

Post attachmentPost attachment
Srihari Sriraman avatar
⟳ nilenso retweeted
Srihari Sriraman@SrihariSriraman·Apr 23

I analyzed 730 SWE-Bench Pro trajectories each for Sonnet 4.5 and GPT-5 and turned them into “trajectory shapes”: when they start editing, when they stop, how much they verify, how many steps they spend understanding vs doing. They have very different work habits.

Post attachment
Srihari Sriraman avatar
⟳ nilenso retweeted
Srihari Sriraman@SrihariSriraman·Apr 22

This was a great opportunity to bridge our industry work at @nilenso with academic research at CMU, and I am very grateful to the co-authors @heathermiller (CMU), Michael Isaac (CMU), and @AtharvaRaykar (@nilenso). We will be in the Bay Area for the conference soon. If you are building AI tooling or want to talk shop about context engineering, we would love to connect.

Drew Breunig avatar
⟳ nilenso retweeted
Drew Breunig@dbreunig·Apr 1

My takeaways from scanning the Claude Code code for ~45 min this evening: 1️⃣Harness engineering is hard. There's a lot of hard won knowledge in here and plenty of diagnostics to keep the feedback flowing. 2️⃣Harnesses and prompts smooth out model quirks. @SrihariSriraman and I covered this last month, but good to see it verified here. So many conditionals based on model types and specific contexts to deploy to mitigate model weirdness. 3️⃣So much of this is CLI app boilerplate. Fully expect a tool like @badlogicgames's pi to be the foundation for any CLI agent being built today. I talk about the last point, the opportunity for shared foundations, in a post today: dbreunig.com/2026/03/26/winc…

Srihari Sriraman avatar
⟳ nilenso retweeted
Srihari Sriraman@SrihariSriraman·Apr 1

Did you know claude code has "model counterweights"? These are patches in the system prompt that exist to balance model biases. These weren't visible earlier, but the leaked code has @[MODEL LAUNCH] annotations that call them out explicitly.

Post attachment