←
Braintrust962 posts
Braintrust avatar✓

Braintrust ✓

@braintrust

Active observability for agents in production.

◷ Joined August 2023▧ 393 Photos and videos
Braintrust avatar
Braintrust@braintrust·Sep 3

Your team deserves more from your agent observability platform - a single, connected place for instrumentation, investigation, and measurement, enhanced with intelligence. In Braintrust, you can start with an open-ended prompt about agent behavior and carry the investigation into code, evals, automation, and monitoring. New tools like Patterns and Debugger make it easier to find recurring behavior, get to root causes, and act on what you learn. The context you build in one step automatically follows to the next. Work in the platform with an enhanced Loop experience, or move to your coding agent of choice. Read more → braintrustdata.link/agent-ob…

4:20
Braintrust avatar
Braintrust@braintrust·19h

Hebbia builds agents that automate outcome generation for the PowerPoints, diligence memos, and spreadsheets that finance teams rely on. With Braintrust they can run a new model through their automated eval process and immediately understand the cost and quality implications. Read more → braintrustdata.link/hebbia

3:25
Braintrust avatar
Braintrust@braintrust·Nov 16

We evaled AWS Bedrock Knowledge Bases. Here's what we found: - Bedrock returned a chunk from the right page 95–96% of the time, but only 73-78% of the chunks could answer the question. - Adding deeper retrieval didn’t reliably improve the final answer. - Depending on the model used to generate the answer, task factuality ranged from 33% to 66%. - Bedrock thrives on direct questions but struggles with comparisons, multi-part questions, and precise role or relationship questions. Learn about the nuances of evaluating an end-to-end RAG pipeline → braintrustdata.link/amazon-b…

Post attachment
Braintrust avatar
Braintrust@braintrust·Nov 13

What's new: - Trace tool calls, prompt requests, and resource reads to debug your MCP server - Define your Braintrust configuration as code with the Braintrust Terraform provider - Focus Patterns on the problems you care about by updating Pattern rules - Trace OpenCode 2 sessions and see images attached to prompts in Muse Code traces through the bt CLI - Translate thread messages and span names into English in desktop Chrome or Edge, without using model credits Read more → braintrustdata.link/whats-ne…

3:57
Braintrust avatar
Braintrust@braintrust·Nov 11

Standardize Braintrust setup across teams with Terraform. You can define projects, permissions, and service accounts as code. Reuse configurations as teams and environments grow, tailor access for each team, and review changes through your existing version control process. Already using Braintrust? Bring existing resources under Terraform management without recreating them. Get started → braintrustdata.link/terrafor…

Post attachment
Braintrust avatar
Braintrust@braintrust·Nov 9

Read a recap of this online workshop with Q&A →

0:23
Indragie Karunaratne avatar
⟳ Braintrust retweeted
Indragie Karunaratne@indragie·Nov 6

We set up MCP tracing + send_feedback tool + Patterns for our own MCP server and it immediately started surfacing useful information on how people are using the MCP server, what's missing, and what's broken.

Post attachment
Braintrust avatar
Braintrust@braintrust·Nov 6

Even if you write good evals, there are always edge cases. Users might try to solve a coding problem with an agent for tracking a pizza delivery.

0:38
Ankur Goyal avatar
⟳ Braintrust retweeted
Ankur Goyal@ankrgyl·Nov 5

If you are not already, you should consider tracing your MCP server. With a little bit of instrumentation, you can get agents to tell you a lot about why they're talking to you, and what issues they're hitting.

Post attachment