←
Harrison Chase17,953 posts
Harrison Chase avatar✓

Harrison Chase ✓

@hwchase17

@LangChain Always hiring: langchain.com/careers

◷ Joined July 2014▧ 706 Photos and videos
Harrison Chase avatar
Harrison Chase@hwchase17·18h

trace data contains a ton of valuable insights to mine, but also a lot of sensitive data you need to manage carefully these three new controls stack up nicely to enable responsible trace governance: - by default, trace everything to langsmith, but restrict access to inputs/ouputs. this lets people see number of calls, tools getting invoked, etc; but restricts any sensitive info - when a customer complains, get their permission and enable a small set of engineers to see inputs/outputs for their bad traces sounds simple/basic/boring, but security and governance is really important to get right for data like traces which can contain sensitive info!

1:16
LangChain avatar
⟳ Harrison Chase retweeted
LangChain@LangChain·19h

We’re adding 3 new controls in LangSmith to help your team manage sensitive trace data. 1️⃣ Control who can see trace inputs and outputs 2️⃣ Manage team-by-team permissions with user groups 3️⃣ Set trace-level access policies using metadata

1:16
Sydney Runkle avatar
⟳ Harrison Chase retweeted
Sydney Runkle@sydneyrunkle·20h

every agent i use now is "proactive", meaning it responds to live events and can schedule its own work, rather than waiting for an ad hoc prompt these are hard to design well, and the two key components are memory and ux documented some lessons learned while building!

Caspar avatar
⟳ Harrison Chase retweeted
Caspar@caspar_br·Nov 16

Humans fall for scams. Agents do too. So how do you keep your data safe when your agent browses the web? In Managed Deep Agents you can now give the browser an allow list of hosts. If a page tells the agent to go anywhere else, the built-in proxy refuses the request Stay safe!!

Post attachment
Sydney Runkle avatar
⟳ Harrison Chase retweeted
Sydney Runkle@sydneyrunkle·Nov 16

you heard it here first folks! as we ask agents to work on bigger problems (goals) the infrastructure around them needs to change to support

Liam Bush avatar
⟳ Harrison Chase retweeted
Liam Bush@LiamBush20·Nov 16

In six months, agents will be “always-on employees" capable of owning work end to end. Agent governance will be the most important aspect of harness design going forward.

Viv avatar
⟳ Harrison Chase retweeted
Viv@Vtrivedy10·Nov 16

big part of “being more ambitious with agents” is putting them in charge of goals my quick view on this shift is “i want the world/my life to look like this” -> let’s go achieve this this should persist over time - go research agent benchmarks ❌ - you’re in charge of our benchmarks - this means daily research covering everything about new AI benchmarks, our goal is to consistently build better benchmarks so orchestrate work to achieve this ✅ there’s a clear tension here between how good agents are at filling in nuance with useful decisions vs how good we are at specifying goals in many domains, any room for agent choice is often slop -> but we need to build systems that allow us to specify goals which are somewhat fuzzy and agents have the tooling and context to achieve them in a way where they can self verify work it’s worth thinking about Task vs Goal in prompting, especially as we move to a world of persistent agents

Harrison Chase avatar
Harrison Chase@hwchase17·Nov 16

as models get more capable, we see them moving from doing tasks towards pursuing goals this is changes what an agent does, what it needs, how its built, and we interact with it this blog covers what we see happening

Sydney Runkle avatar
⟳ Harrison Chase retweeted
Sydney Runkle@sydneyrunkle·Nov 16

the most important type of context is domain knowledge and taste this is often delivered in the form of feedback, so review UX needs to be delightful this enables agents to continually learn and improve, becoming increasingly a) useful and b) autonomous

Ajitesh Shukla avatar
⟳ Harrison Chase retweeted
Ajitesh Shukla@vitalflux·Nov 15

Langgraph Workflow or deep agent? For our procure-to-pay (P2P) agent on @LangChain , the answer was: a workflow on the outside and an agent on the inside, built so the inside can grow. What we drew on: • Anthropic's "Building effective agents": use workflows for well-defined tasks, and add agency only when you need the flexibility • LangGraph: mix deterministic steps and agentic steps in one graph, with a full agent as just another node How we applied it: 1/ Deterministic shell Load context → agent core → exit gate → write back. This is plain code and the same every time. P2P operations gets one auditable path. 2/ Agentic core behind one interface AgentCore.run(context) → Resolution Today it's a create_agent loop: the steps are known and the context is small. When cases get messy (several invoices, a dispute, a question for another team), create_deep_agent drops in with the same tools and skills. 3/ Guarantees in code, not prompts Prompt rules are probabilistic. Rules checks and an independent reviewer run in the shell, outside the model, so the core can't skip them, however it plans. 4/ Steps as skills, not graph nodes A new scenario becomes a new skill, not a graph redesign. The reverse also exists: some teams run an agent on the outside and call locked-down workflows as tools for risky actions like refunds. Pick the layer by the stakes. For us, every case must clear the same audit, so the deterministic part sits on the outside. Rule of thumb: if you can draw the flowchart, it's a workflow. Put the agent where you can't, and keep the guarantees where the agent can't touch them. #AIAgents #LangGraph #LangChain #AgenticAI

Post attachment
Ajitesh Shukla avatar
⟳ Harrison Chase retweeted
Ajitesh Shukla@vitalflux·Nov 15

Building an AI agent? Write the eval runner before the agent. We're building an agent using @langchain deep agents that answers vendor invoice queries in procure-to-pay (P2P) workflow, and the eval runner is the first thing we're writing. Why it comes first: • You have to define "correct" before you write a single skill • Every skill, model or tool change is re-scored on the same golden dataset • You improve the agent against numbers, not gut feel So the agent depends on Ports, small Python interfaces, and we plug in mock adapters that return fixed test data: class InvoicePort(Protocol): def get_status(self, invoice_id: str) -> InvoiceStatus: ... class MockInvoiceAdapter: def get_status(self, invoice_id): return FIXTURES[invoice_id] The agent can't tell it's talking to a mock. What you get: ✅ Repeatable evals that give the same result every run ✅ Agent work starts on day 1, not when access is granted ✅ Real adapters replace the mocks later, with no change to the agent or the evals ✅ The frozen mocks become your regression suite Evals first. Ports second. Agent third. #AIAgents #EvalDrivenDevelopment #Python #AgenticAI #LLMOps

Post attachment
Caspar avatar
⟳ Harrison Chase retweeted
Caspar@caspar_br·Nov 14

Cool lesson in agent design from Actively. How you structure your agents shapes how much they improve over time They shifted from one agent per customer to one per class of work, so lessons carry across customers and fewer corrections need repeating

Post attachment