
What are you testing right now that wouldn't have been testable a year ago, and what did you stop testing? linkedin.com/pulse/what-do-y…

CEO @ speero.com Experimentation & User Research (now with AI)

What are you testing right now that wouldn't have been testable a year ago, and what did you stop testing? linkedin.com/pulse/what-do-y…

Not saying A/B is dead. The human web still holds most of the revenue. 100% A/B testing in 2026 is out of position, 0% is too. It's a portfolio, and when you can't run a clean test you climb confidence with cheaper proxies (simulation, research, narrow trials).

So what do you test? 4 buckets I'm working with: the eval layer (what the model sees and says about you), the data and content layer, the handoff moments where an agent pauses to ask or confirm, and decisions built from messier evidence than a clean A/B.

This isn't hypothetical for me. In a normal week I log ClickUp hours, update deals in Attio, and fix SEO issues in Webflow without opening any of them. I describe a test idea by voice, Claude interviews me on study design and writes the Airtable record.

A/B testing assumes the user opens something. A page. Pixels you can rearrange. That assumption is breaking. Agents mediate. No DOM. No session to randomize. A/B was always one method inside a bigger thing. What do you test when there's no UI?

Where does that function report in your org today... product, marketing, data, a CGO? Or does it not exist yet? linkedin.com/pulse/unit-test…

So the bottleneck moved from "how do we ship more" to "how fast do our beliefs actually update from real signal." That's a program-level question. No single CRO or product team owns it in the old org model. IMO that's the growth function's job.

Leah's evidence that we're turning a humility tool into a confidence machine: once NIH trials had to pre-register the primary metric, the share reporting a significant win went from 57% to 8%. Now AI makes picking the metric and narrating the win 10x faster.

Why it matters: that shift rearranges who runs experimentation, where it reports, and what infra it needs. At Circus London, Leah Tharin asked a room of very experienced practitioners who works somewhere with a real dedicated growth function. Basically one hand went up. Mine.

For a decade we tested artifacts. Buttons. Headlines. Pages. AI collapsed the cost of producing those to roughly zero. The unit of test moved up: models that generate variants. Marcel Toben: "AI collapsed the cost of change. It did not collapse the cost of being wrong."

We're prototyping this at Speero now. If you've been burned by an AI form filler that just made worse forms, where did it fall over? linkedin.com/pulse/intake-ag…

It takes 3 ingredients. Context (strategy, metrics, past tests), methodology (what good looks like in your shop), and connection (it writes to the same Airtable/Jira/ClickUp the program runs on). In Claude that's Projects, Skills, and MCP. Copilot and Glean, same recipe.

Same sentence typed into an agent instead. It doesn't file the idea, it asks: what's the construct, endowed progress? We have 3 candidate primary metrics. This audience overlaps the live pricing test, want to coordinate or re-segment? The PM walks away with a better brief.

Why this matters: the PM has the idea, the org has the strategy, the metrics library, the live tests, the past results. The form doesn't connect them. So the reviewer re-introduces that context by hand (if they're senior enough to know it), a translation tax that compounds.

A PM types "test a progress bar on onboarding" into a form. The system already knows the adjacent test was flat, the audience overlaps a live pricing test, and activation isn't this quarter's goal. The form tells them none of it. Intake shouldn't capture. It should interview.

I'm biased toward velocity, so push on it. Have you ever written down what a false positive costs you, in dollars, vs what a missed winner costs you? Full piece: linkedin.com/pulse/two-kinds…