←
Distributional157 posts
Distributional avatar

Distributional

@dbnlAI

Deploy AI with Confidence

◷ Joined September 2023▧ 82 Photos and videos
Scott Clark avatar
⟳ Distributional retweeted
Scott Clark@DrScottClark·Sep 1

Better tools to do better science faster is the rising tide that lifts all boats toward human abundance. At @TalariaSci we're building a self-reinforcing loop for better discovery greater accuracy in physical AI. Learn more here: talariasci.com/blog/the-righ…

PME avatar
⟳ Distributional retweeted
PME@itsyourcode·Aug 4

Timely reading for anyone building agent harnesses in domain specific / OOD problem spaces from @DrScottClark

Post attachment
Scott Clark avatar
⟳ Distributional retweeted
Scott Clark@DrScottClark·Jul 29

I wrote a post about all the mistakes I made as a second time founder and my journey pivoting @dbnlAI to @TalariaSci. Hopefully this helps the next founder that tricks themselves into thinking the second time will be easier (you should do it anyway!). scottclark.io/blog/make-all-…

Distributional avatar
Distributional@dbnlAI·Jun 15

DBNL helps you make sense of the patterns in your AI agent behavior, detecting where there are deviations and then providing automated recommendations so that you can build an application that behaves like you want it to. Get started: distributional.com

2:54
Distributional avatar
Distributional@dbnlAI·Jun 12

Kick off your AI analytics flywheel with DBNL. DBNL transforms traces into actionable insights, ensuring teams can continuously fix and improve their agents with the right evidence in production: dbnl.io/cPRvZf

Post attachment
Distributional avatar
Distributional@dbnlAI·Jun 10

Is your AI agent actually calling a tool when it claims to? Scott Clark explains one of the most common anti-patterns he sees with multiple models today, where the agent gets "lazy" and will say it called a tool but the traces prove otherwise. Learn more about how DBNL's analytics helps teams catch dark patterns: dbnl.io/iMRLJz

1:34
Distributional avatar
Distributional@dbnlAI·Jun 9

Every day, DBNL users get fresh insights on their most recent production data. These insights drive the development of new offline metrics to guide reward functions, evals, or context for agents. They can be used to generate new test cases that are run pre-production on agents or in a continuous “integration testing” style setting. And they create new metrics to be monitored real time by more traditional observability tooling. Learn more about how DBNL analytics generate actionable insights: dbnl.io/cPRvZf

Post attachment
Distributional avatar
Distributional@dbnlAI·Jun 8

Your agent handles 50,000 traces a day. Accuracy is 87%. Error rate is stable. To your dashboard, everything is fine. Meanwhile: → A retrieval path is silently returning 90-day-old documents to users asking about recent policy changes. → A prompt tweak from last Tuesday shifted tool-calling order in a way that tanks task completion for multi-step workflows by 12%, for four hours, every Tuesday. → A whole topic category the agent answers confidently is wrong 40% of the time. None of this shows up in your metrics. You find out when a user complains, or when satisfaction scores quietly erode over a month and you can't explain why. Between your monitoring dashboard and your trace viewer, there's an analytical gap. Learn how to convert agent traces into behavioral insights with DBNL. Connect your OTEL traces and get insights in minutes: dbnl.io/Luuawb

Post attachment
Distributional avatar
Distributional@dbnlAI·Jun 4

Distribution analysis answers the questions that summary metrics can't. DBNL's distribution comparison can help you pinpoint the issue—this is especially helpful when evaluating experiment variants: dbnl.io/CjZ9X9

0:26
Distributional avatar
Distributional@dbnlAI·Jun 1

AI agents are complicated, but catching and fixing issues with them is easy with DBNL. From pre-deployment evals that fail to perform in production to insights on user intent and more, DBNL helps teams identify issues faster: dbnl.io/uVtqmf

Post attachment
Distributional avatar
Distributional@dbnlAI·May 29

When you sample traces, you’re making a bet that the interesting patterns are distributed randomly across your data. They’re usually not. Random sampling will systematically misses low-frequency, high-impact patterns. Where does your organization fall on the observability maturity scale? Read and find out: dbnl.io/i00yOD

Post attachment
Distributional avatar
Distributional@dbnlAI·May 27

Your P95 latency doubled after a deployment, but your average cost-per-call looks fine. Which one do you trust? This is the core tension in LLM agent observability: summary metrics tell you something changed, but not what changed—or for whom. For example, a bimodal latency spike (where half your requests are fast and half are stuck) looks identical to a uniform slowdown when you compress it to a single number. DBNL's distribution comparison unlocks the details. Here's how it works: dbnl.io/CjZ9X9

0:20
Distributional avatar
Distributional@dbnlAI·May 20

AI agent analytics insights and recommendations—automated. DBNL automatically turns logs into action items, helping you identify where your agents could use some tweaks. Learn more: dbnl.io/IvoBGJ

Post attachment
Distributional avatar
Distributional@dbnlAI·May 19

Agent breaking in production? There are 3 types of issues that DBNL's analytics can help catch before they become a bigger problem: 1. Offline, pre-deployment evals that fail to perform out of sample in production 2. Insights on user topics, intent, and input patterns 3. Issues in the complexity of agent behavior dbnl.io/uVtqmf

Post attachment
Distributional avatar
Distributional@dbnlAI·May 18

Are you missing key information about your AI agents? Here are the 3 failure modes that are invisible to aggregate metrics: → Behavioral drift by segment → Unknown-unknown behavioral patterns → Gradual degradation vs. hard failures Take our quiz to see where you land, then learn how to close the gap with DBNL: dbnl.io/arGD1z

Post attachment