←
/sesh/null810 posts
/sesh/null avatar

/sesh/null

@nerdsane

VP, cognitive systems & research @datadoghq | minimalist { engineer | athlete | artist } | opinions-my-own

◷ Joined December 2017▧ 37 Photos and videos
/sesh/null avatar
/sesh/null@nerdsane·Nov 14

I speak very little Telugu with our 3yo and that’s why Rita built this so he doesn’t lose it 😀

Hyper-personalized software is also hyper-personalized education. Why do school districts matter, for example? I think you'd pick one for cultural and intellectual fit. But intellectually, when you go through a curriculum with a group, everyone could get pulled to the baseline. Kids above it might get disengaged cause they're bored, kids below it cause the gap is too big. And even at the right level of complexity, the way it's taught might not suit everyone. I changed schools five times growing up (post-soviet, a very different education model). We accepted many limitations and "turned out fine", but so much more is possible now. I've been thinking about this as my son nears school age. With AI you might not need to pick by intellectual baseline anymore, cause there wouldn't be one. The knowledge part of education would become a game with infinite levels, where each "boss" you need to defeat is the kind that motivates you personally the most. No limit to imagination. The social aspect and the humanity of it, the interpersonal part would be the core value of group education AI can't (shouldn't) replace. So you'd pick based on that. We're making this happen at home. My son just turned 3, and he has a tablet for when we travel. I curate the games on it, but he gets bored of the same ones, and in a household where we speak Russian, English and Telugu, educational games in the right language can be hard to find. I'm also introducing him to Mandarin, because I spent a lot of time studying it myself and want him to be familiar with it too. So I made him a personal game: a world of islands with stories he listens to, and games woven into them (letters, counting, finding animals), and it grows as he uses it. He got right away that it changes, and started "prompting" it to change too. It kept showing him the same word, and he talked straight to the tablet: "no, I've seen this word, it should say (his name) there instead, and you add Shere Khan, Mowgli, Bagheera too. And wolves". (He's in his Mowgli/jungle phase currently). Basically he told that to the device, and I just translated toddler-speak to Claude and it built that. He knew to do that at 3 (I was a bit shocked tbh) because he hears me talking to Claude a lot + others in our family talk to agents. He'll say Claude is Gemini's friend, because one person talks to Claude and another talks to Gemini, which means they're kind of the same thing you talk to, so they must be friends. This new generation picking it up is fascinating. This app is made with @UniKonDev, which I'm building for personal use cases like this. I also use it to: organize family to-dos, build UniKon itself with my agents, collect design inspiration, monitor and research the housing market, collect my son's quotes, etc. @bot, @muse, Claude and other agents can build the same app. It has a backend that keeps his progress and the game's rules, and it runs in a sandbox (@tensorlake) that's only awake when he's playing. So all of my son's interactions with it are recorded: if he gets stuck somewhere, skips something, or already knows the answers, it's all in the state. I can ask the agents building it to extend it or add new content that will challenge him now, or background agents can just do it and I only curate. And the more this world is built out, the easier it is for agents, because it's collecting his playing history (the kind of things he liked and didn't like). I feel a kinship with what @lionel_mora is building here. I can also share the app with someone else, and UniKon keeps the progress, personal details (and even the look of the app) unique to each user. Or they can just copy my app and have their agent change it however they want. You can build apps like this easily with UniKon.

0:58
/sesh/null avatar
/sesh/null@nerdsane·Nov 13

Agree with the specific approach and the overall thesis of the need for higher-order programming abstractions.

I strongly believe (hope?) humans should design and maintain software (likely in natural language). As this transition happens, we must also increase the level of abstraction used for verification (aka tests) Transcript Tests can help: andrew.nerdnetworks.org/blog…

/sesh/null avatar
/sesh/null@nerdsane·Nov 7

Have been calling this Directed Evolution

signal is a great word for it. Recently I encouraged folks to stop thinking in terms of "loops" and more in terms of: "forward pressure" - what causes work to start (crashes, bugs reports, performance issues, +100 things) and "backpressure" - how does an agent hill climb against an outcome (linters, tests, ai review/risk judge, canaries, feature flags, +100 things) looping these toward a desired state rather than an outcome / artifact is a powerful mental model - forward pressure increases autonomy in what gets attention, backpressure increases autonomy in getting to a super high-quality solution 9 months ago, loops like ralph were primarily used to go from point A to point B in terms of functionality, but this is just one degenerate case of a more generic control loop: - read desired state (e.g. no open bugs, or DB perf above X metric, OR "feature Y shipped", OR "all linter warnings resolved or explicity ignored") - read state of the world (query systems, run linters, etc) - decide if anything needs to be done (compare to policy / desired state) (optional: approve decision) - do the thing (code change, infra change, docs change, process change) (optional: approval action) - repeat shouts out @0xblacklight for the original framing here at AIE back in june. same logic works for evolving the control loop itself

George Pîrlea avatar
⟳ /sesh/null retweeted
George Pîrlea@GeorgePirlea·Nov 7

Veil 2.0 is out and I’m very proud of it. @zqy1018 spearheaded this release and did an amazing job. Anything you’d use the TLA+ Toolbox for, we now support in a fully foundational way and with better model checking performance! Give it a try!

Veil 2.0 is out: verify all your distributed protocols in Lean 4. New: verified model checkers for safety (2–20x faster than TLC) and liveness, LTL proof mode, TLA+ → Veil LLM skill, PlusCal-style DSL. Great work by @zqy1018 and external contributors! veil.dev/

Post attachmentPost attachmentPost attachment
/sesh/null avatar
/sesh/null@nerdsane·Nov 7

One of the things I built for myself (amongst others) on Unikon is a chess boxing trainer that I share with my trainer. He talks to it through his @Muse and I with @bot

Post attachmentPost attachmentPost attachment

You want personal software to keep evolving as you use it, and as your agents use it for you. I’m building @UniKonDev with this in mind. I personally use several agents (@bot, @Muse, Claude), so when I have bot build an app that pulls in design inspo from galleries and lets me curate and keep the best, later when I ask Claude to design something for me, it needs to know that context too. Then, personal software is also shared with your circle. My husband has a @bot, my son's nanny has ChatGPT with dots. So I asked an agent to build us a Unikon app where we share the calendar of my son's activities, quotes of things he says and photos too, and whenever one of our agents need something they can’t do yet, they add it.

Josh avatar
⟳ /sesh/null retweeted
Josh@JoshPurtell·Nov 7

Have you seen temper by @nerdsane ? I’ve used it to help agents build state machines for e.g. monitoring processes also and it works really smoothly for it ime, and allows them to build really powerful experiences like e.g. custom progress bars, streaming log info, etc

UniKon avatar
⟳ /sesh/null retweeted
UniKon@UniKonDev·Nov 6

Everyone has their own agent now. With UniKon, they build one app together and coordinate through it. Say, you're planning a trip: you use Claude, your friend uses @bot, your mom - ChatGPT, and everyone's agents work on the same app, planning where to stay, sharing photos with each other (your own little feed) and adding a new page whenever someone needs one. UniKon is built on @tensorlake sandboxes, uses @datadoghq for observability, apps run in Temper runtime by @nerdsane.

1:28
/sesh/null avatar
/sesh/null@nerdsane·Nov 5

Had a great time on the pod, thanks for having us @meganreyno

My timeline is full of RL. RL environments, RL gyms, data labeling companies. Startups scaling to $1B in revenue in months. What is going on? Post-training seems like the new frontier. Reinforcement learning is the emerging preferred method, with an ecosystem worth $Bs growing around it. But how does it work? And how much does it really add to good pre-training? On Ep 3 of Technically Speaking from @readtechnically, a rare conversation from both sides of the RL stack: @slippylolo (co-founder & CEO of Adaptive ML, now at @datadoghq ) and @nerdsane (Datadog): → Why a monitoring company bought an RL startup: "At its heart, this is an observability problem." → Whether your startup should train its own model: "for 99% of people it doesn't make any sense to do that" → Creating synthetic training data: "You cannot create data ex nihilo and hope for the best." → What actually makes for great post-training: "it's mostly a question of high quality execution"

0:34
arni avatar
⟳ /sesh/null retweeted
arni@arni0x9053·Nov 4

In formal verification there are two axes: checking vs proving, and doing either on a model vs the actual implementation. For checking, formal modeling languages I use : TLA+, @quint_lang, Stateright. These model the system design and check that in all possible states, as the model defines them, undesirable things don't happen (with invariants) and desirable things do (liveness). "All possible states" means within the bounds you set, like 3 nodes and 2 retries. Checking explores every one of those states exhaustively, but doesn’t check what happens with 4 nodes for example. I'd say it helps to be able to read at least one of those and understand concepts like invariants, liveness and fairness, if we're relying on the formal model to be the backbone of the whole design. Claude will smoke them out fine, and even if you don't wanna read it or write it yourself, conceptually being able to grill Claude and understand what it's explaining is important. Then an agent needs to keep the implementation aligned to the spec. (Stateright is the exception: the model is Rust actor code you can also run, so the model and the implementation can be the same thing). @poteto's pstack has a skill, /create-formal-verification-skill, that uses TLA+ to find bugs in existing codebases. I'd try that and then ask Claude to teach you what it did and why. Then there's proving. Checking covers the states it explored. A proof covers all possible states, because it proves that something is true, like you'd prove a theorem in algebra.
The unlock here with agents is that once you define the rules, they can write the mathematical proofs of them about a model or code (which is extremely labor intensive for a human). Tools and languages that do proving: Lean, TLAPS, Verus, @bendlang (@VictorTaelin), Z3. Lean and TLAPS prove a model, so you need to keep the implementation aligned again. Verus can do either: it can prove a Rust implementation directly, or you can use it to write a model. In @bendlang you declare laws about your program and prove them in the same language, and the compiler checks the proof before the code runs. So the proof is about the actual code. Z3 is the solver underneath a lot of these: you give it logical constraints and it either finds a counterexample or shows there is none. Verus is built on it and TLAPS can call it. You usually don't write Z3 yourself. I'm personally building @UniKonDev, a multiplayer substrate where agents build apps, and every app is made of executable state machines. It runs on Temper, a runtime by @nerdsane, which executes those state machines and refuses moves that break the app's rules. We’re combining checking for exact counterexample traces with proving for what's outside the check's bounds.

Alperen Keleş avatar
⟳ /sesh/null retweeted
Alperen Keleş@Keleesssss·Nov 3

We seriously should not keep saying this. Pretty soon some "formally verified" software is going to have bugs, and people are going to start questioning what value formal verification really has.

There’s no reason for exploits and bugs to exist. Let’s build (and verify) together!

Jonathan Parisot avatar
⟳ /sesh/null retweeted
Jonathan Parisot@JoParisot·Nov 3

Bits is getting more accurate and faster At the same time, we're making it more affordable so that it makes sense to have bits being the first responder to every single alert The average cost for an incident investigation was just reduced by 40% and we'll keep on doing more

Post attachmentPost attachment
arni avatar
⟳ /sesh/null retweeted
arni@arni0x9053·Nov 3

Glad I’m not the only one feeling this. Adversarial review (especially multiple rounds) seems to inevitably drive agents off the cliff into the over-engineering abyss. It’s formal verification -> deterministic simulation -> agents using the product as if they are users

maybe a hot take but i now consider adversarial reviews mostly a waste of tokens. far better to use those tokens on empirical combined with formal verification. i have updated pstack to use the interrogate skill less, and eventually i think i may remove it altogether i have found very little value with code review in the abstract, agents tend to just get lost in the nitpicks and yagnis

/sesh/null avatar
/sesh/null@nerdsane·Oct 31

Agreed. @arni0x9053 nailed it here.

lauren@potetoOct 30

we will need to rebuild and rewrite much of software to adapt to a new, faster pace. code used to arrive one drop at a time. soon, if not already, code will be written as if it were a continuous stream, like a violent river twisting and turning as it pleases. this is not a new thing: big tech cos have had to deal with this with the enormous scale of their engineering teams. but now ai has brought an incredibly elastic and capable workforce wielded by every engineer. this is your problem now, and the only way to adapt is to evolve your systems so agents - even and especially the ones with little context about the right way of doing things - can thrive. make the easy thing the right thing. your codebase is becoming organic architecture. agent contributions grow around the constraints and guardrails you’ve designed. good software engineering matters even more now! and no, the continuous stream of code is not something you have to watch helplessly. be more ambitious and design systems, frameworks, and languages to control the chaos. the time for mourning coding is long over