←
Eric Lin37 posts
Eric Lin avatar

Eric Lin

@EricLin1618

self-improving agents and evals agent hillclimbing @ decagon cs @ harvard

◷ Joined June 2017▧ 1 Photos and videos
Decagon avatar
⟳ Eric Lin retweeted
Decagon@DecagonAI·Sep 22

We put nine leading models through 1,373 battles to find the best balance of quality, speed, and cost on complex enterprise tasks. In our latest results, GLM-5.3 broke the Pareto frontier by achieving frontier-level quality with incredible efficiency, finishing tasks in less than a third of the time than Opus 5. More in the thread. ↓

Post attachment
Eric Lin avatar
Eric Lin@EricLin1618·Jun 10

Excited to share what I've been working on the past few months!

One of the hardest parts of building self-improving agents is proving they are actually improving. That’s why, alongside Duet Autopilot, we built DuetBench: the first benchmark designed specifically for CX agents that learn and improve over time. To evaluate Duet Autopilot, we compared its performance against certified human agent builders and graded both on outcome and methodology across 90 diagnostic investigations from simple metric lookups to root-causing CSAT drops. We also evaluated Autopilot on enterprise agent-building tasks. Starting from messy design documents, it had to build AOPs and tools from scratch, generate simulations, and pass every associated test before a task was considered complete. Autopilot demonstrated an iterative approach to agent building. Rather than solving problems in a single pass, it ran simulations, identified broken branches, repaired the AOP or underlying tool, and repeated the process until the workflow passed. Another notable result was that Autopilot improved the quality of its own test set through self critique, increasing simulation accuracy from 58% to 88% across 520 benchmark runs. As self-improving systems become more common, verified evaluation will matter just as much as model capability. Excited to share the research behind it. Full writeup below. ↓

Post attachment
Daniel Litt avatar
⟳ Eric Lin retweeted
Daniel Litt@littmath·1 Feb 2025

Basically any resource on a difficult subject—a colleague, Google, a published paper—will be wrong or incomplete in various ways. Usefulness isn’t only a matter of correctness.

SkalskiP avatar
⟳ Eric Lin retweeted
SkalskiP@skalskip92·18 Jun 2024

tip 4: few random ones my favorite: adapt to tasks by formatting data in a proper way not by designing custom architecture.

Post attachment
Boston Symphony avatar
⟳ Eric Lin retweeted
Boston Symphony@BostonSymphony·9 Feb 2024

(1/3) With great sorrow, we announce the death of our beloved Music Director Laureate Seiji Ozawa, who passed away on February 6 at age 88 in Tokyo.

Post attachment
Edge Impulse avatar
⟳ Eric Lin retweeted
Edge Impulse@EdgeImpulse·5 Oct 2021

TinyFedTL is the first implementation of federated transfer learning on a resource-constrained microcontroller: https://t.co/GnT7KQwsNM cc: @KavyaKopparapu @EricLin1618

Post attachment
Catherine Yeo avatar
⟳ Eric Lin retweeted
Catherine Yeo@catherinehyeo·27 Apr 2021

Just helped a friend in New Jersey book an appt for her COVID vaccine using a resource found on our site! She's been waiting for *weeks* for an open slot at her local Walgreens. Using covidvaccinesinfo.com took 2 mins to find a slot. Book an appt today & get your COVID shot ⬇️

My grandma's COVID vaccine appt got cancelled today. My family panicked to find another. I used covidvaccinesinfo.com to find a resource to book her a new appt. @EricLin1618 & I built this site to share resources per state to help such situations. Please check it out & share!

Catherine Yeo avatar
⟳ Eric Lin retweeted
Catherine Yeo@catherinehyeo·30 Mar 2021

Thank you @VickyNguyenTV @MichelleTak_ for including covidvaccinesinfo.com as a useful vaccination appointment resource on today's @TODAYshow segment! As more resources emerge, it's key to be aware of your data privacy & which sites are helpful vs malicious. See more here:

TODAY@TODAYshow29 Mar 2021

As people continue to search for vaccination appointments, many are turning to websites and other digital tools that promise to alert you to availability. But how do you know if they’re legit? @VickyNguyenTV shares what you need to know.

4:07
Catherine Yeo avatar
⟳ Eric Lin retweeted
Catherine Yeo@catherinehyeo·22 Feb 2021

My grandma's COVID vaccine appt got cancelled today. My family panicked to find another. I used covidvaccinesinfo.com to find a resource to book her a new appt. @EricLin1618 & I built this site to share resources per state to help such situations. Please check it out & share!

Eric Lin avatar
Eric Lin@EricLin1618·4 Feb 2021

Ever wondered when you’ll be eligible for a COVID vaccine? 🤔@catherinehyeo & I created the COVID Vaccines Info Guide, your one-stop source for trusted info on getting COVID vaccines. Check it out, share it with your friends, & let us know what you think: covidvaccinesinfo.com

Post attachment
Catherine Yeo avatar
⟳ Eric Lin retweeted
Catherine Yeo@catherinehyeo·28 Dec 2020

Thank you @venturetwins for featuring @sesame_call in the latest newsletter on top 10 trends for 2021! We're proud to be building a video- & audio-centric digital dorm for students and excited to keep growing it in 2021 🌟

Post attachment