
"Introduce ambient Al into your workflows and what actually changes hands is authority over attention, which in a company is where the power sits."
✓Founder w/roots in academia. Founder @MIT Cryptoeconomics Lab. Past: Co-Founder & Chief Strategy Officer, Lightspark. Co-Creator, Libra. Head Economist, Meta.

"Introduce ambient Al into your workflows and what actually changes hands is authority over attention, which in a company is where the power sits."

The alternative to a competitive market for AI is grim. That's why open source and open weights matter.

It's not winner-take-all. Nor winner-take-most. But "one of many frontier labs" is a tough roadshow pitch.


The idea that frontier AI labs will absord everything in the economy, including fintech and finance, is untenable That's what MIT Cryptoeconomics Lab founder @ccatalini argued on my new Intent Layer podcast series His claim is that with every major platform shift — the steam engine, electricity, railways, the internet — value went to the holders of complementary assets, not to the inventors of the core technology For AI, those assets are: - Distribution: You don't need the best model to give billions of people a great consumer experience - Chips: Nvidia controls the hardware everything runs on - Atoms: SpaceX controls the physical scale-up, and manipulating real atoms is still quite difficult The moat may not always be who has the smartest agent or best model. Rather it's whoever owns the customer relationship, the license, and the rails that agents need to move money

The bottleneck to scaling is capital. Raising it requires a credible path to appropriability. Appropriability requires owning the complementary assets. The labs own the weights. Weights are not a complementary asset.

Throwing more tokens at vulnerabilities buys a longer patch queue. We need architectures that eliminate whole classes of failure. Cybersecurity is the preview. LLMs are already finding holes in the legal code. @TalBeerySec medium.com/@TalBeerySec/dens…

AI is making intelligence and productivity cheap. Verifying its outputs is now the bottleneck In this episode of my new show The Intent Layer, @ccatalini argues that verification is becoming a business opportunity for companies with knowledge the frontier labs don’t have We discuss why companies may need to do more than sell AI-generated work. They may also have to take responsibility when it’s wrong, potentially backed by insurance We also get into his skepticism about everyday agentic commerce needing stablecoins, and where he sees a more plausible opening for crypto payments 00:00 MIT, Libra and Lightspark 02:00 Libra’s lessons for stablecoin consortia 06:52 Why verification becomes the bottleneck 10:00 Companies as verification businesses 14:00 AI liability and insurance 17:00 Competing with the frontier labs 19:50 Crypto, identity and verification 22:00 AI-enabled attacks and software security 23:40 Does agentic commerce need stablecoins? 25:00 Agents, bank deposits and switching friction 28:30 Who captures the value from AI? Watch the full conversation: youtube.com/watch?v=BPboYpFL…

5/ @AnthropicAI's data shows this is happening faster than we expected. Keeping a human in the loop tells us very little unless that human has the context, attention, and tools to apply their experience and preferences.

Easy to count the attacks they enable. Harder to count the ones they stopped.

3/ ...apparently productive work carries assumptions and preferences that humans never independently evaluated. Runaway risk zone: the work is completed, the expert signs off, and measured output rises. But whose experience and preferences shaped the result? ↙️

Turns out better verification tooling was hiding in plain sight all along…

Execution is getting cheaper faster than the human ability to actually verify the work and retain control over key decisions. Many specialists will simply lose the skill of analysis and start relying more heavily on the agents’ work and explanations. Agents make mistakes just like people, so the ability to analyze and verify errors made by agents should remain one of the key skills for specialists.

2/ Experts followed their agents’ randomly assigned preferences on 86% of choices. Many then regarded the resulting recommendation as their own. This is the failure mode behind the Trojan Horse externality...

6/ Verification infrastructure is how we counter all this. So is "attention ergonomics" (see @sschillace's piece, H/T @livjenkss). 🚨 We can’t safely automate what we can’t verify, and we can’t verify what no human is paying attention to.

Almost every dollar of AI safety funding today goes to understanding, testing, containing or defending the model itself, on the assumption that the danger is a model that misbehaves. But safety money only covers the first half of an AI output’s life, and almost no funder has dug into these questions: - Did anyone actually check this output? - Whose judgment is it, the person's or the agent's? - When it turns out to be wrong, who pays? Funders mistakenly leave them out for a few reasons imo: - An honest agent that steers an expert into adopting its defaults isn't misbehaving, then it falls outside the frame for funders. - Checking an individual output before someone acts on it has no established name in the grant vocabulary. - Funders focused on existential risk filter for direct paths to catastrophe. - The disciplines that know this problem best, human factors, cognitive psychology, etc. rarely apply to AI safety funders, and fieldbuilding mostly trains machine learning researchers. - It looks like a product problem (i.e. better review interfaces and audit logs), but it's not. There are plenty of nonprofits to be built here, that would let us see, measure and price what happens when a person accepts an agent’s work.