←
TRISHALA871 posts
TRISHALA avatar✓

TRISHALA ✓

@hitrishala

tinkering around at evidentmachines.com (yc f26) prev @googlelabs @mongodb eecs @ucberkeleymet

◷ Joined July 2021▧ 30 Photos and videos
TRISHALA avatar
TRISHALA@hitrishala·16h

Welcome to the real world. Today we’re introducing FrontierEE Bench, the first benchmark to test frontier models on electrical engineering tasks at a physical lab bench. Real-world engineering benchmarks should involve real hardware. At our lab, models control instrumentation, run experiments on physical hardware, and decide what to test next. The best models scored 36.7% across a range of EE domains including mixed-signal circuits, power systems, & electromechanics.

0:05
TRISHALA avatar
TRISHALA@hitrishala·16h

The next frontier for AI is autonomous hardware R&D: models that can design, test, diagnose failures, and revise until a physical system works. That capability is learned through real experiments. We’re turning the failure modes we find into training data and RL environments grounded in physical lab work. We’re making physical experimentation part of how AI learns, starting with the electrical engineering that powers the economy – and AI itself. Read more about our benchmark results at evidentmachines.com/frontier…

TRISHALA avatar
TRISHALA@hitrishala·16h

Models passed 100% of these tasks in simulation. On real hardware, the average pass rate was 29.3%. Across 10 real-world tasks, models built open circuits, ignored stored charge, bypassed current limits, and calculated component properties from incomplete models.

0:21
TRISHALA avatar
TRISHALA@hitrishala·16h

The real world is messy. Measurements are noisy, components are defective, and hardware behaves in ways simulation never captures. We built a custom harness that lets AI run autonomous experiments at our SF lab. Models send commands to instrumentation, request wiring changes from a human operator, and iterate on physical measurements. We record their decisions and experimental outcomes. This is the future of autonomous hardware research and development.

0:15
TRISHALA avatar
TRISHALA@hitrishala·16h

The next frontier for AI is autonomous hardware R&D: models that can design, test, diagnose failures, and revise until a physical system works. That capability is learned through real experiments. We’re turning the failure modes exposed by FrontierEE Bench into training data and RL environments grounded in physical lab work. We’re making physical experimentation part of how AI learns, starting with the electrical engineering that powers the economy – and AI itself. Read more about our thesis at evidentmachines.com

TRISHALA avatar
TRISHALA@hitrishala·16h

Models scored 100% on these tasks in simulation. On real hardware, the average pass rate was 29.3%. Across 10 real-world tasks, models built open circuits, ignored stored charge, bypassed current limits, and calculated component properties from incomplete models. Read more about our benchmark results at evidentmachines.com/frontier…

0:21
TRISHALA avatar
TRISHALA@hitrishala·16h

The real world is messy. Measurements are noisy, components are defective, and hardware behaves in ways simulation never captures. We built a custom harness that lets AI run autonomous experiments at our SF lab. Models send commands to instrumentation, request wiring and probe changes from a human operator, and iterate on physical measurements. We record their decisions, instrument readings, and experimental outcomes. This is the future of autonomous hardware research and development.

0:15
TRISHALA avatar
TRISHALA@hitrishala·Nov 17

i heard that a really awesome benchmarking startup is launching a super novel first-of-its-kind benchmark tomorrow. it’s just a rumor. idk!

TRISHALA avatar
TRISHALA@hitrishala·Nov 17

hardware has always been expensive enough to force some discipline about what gets built. AI loosens that constraint for design - but it doesnt change the cost of board bring up/fixtures/bench time EEs who benefit from AI the most will be ruthless at cherrypicking designs and deciding which of them are actually worth taking into manufacturing

TRISHALA avatar
TRISHALA@hitrishala·Nov 16

it’s crazy how papercut-y the physical world is. we use the most advanced simulators we can get our hands on to pre-screen our EE benchmark tasks & models that pass flawlessly in simulation get bogged down by the randomest tiny papercuts in the real world. there are so many of these papercuts to keep track of across different domains of EE that it’s hard to imagine simulators ever capturing all of them accurately enough to replace building + testing the actual thing

TRISHALA avatar
TRISHALA@hitrishala·Nov 14

teaching ai about the physical world. unfortunately this is the physical world

Post attachment
TRISHALA avatar
TRISHALA@hitrishala·Nov 9

the faster we can design hardware, the more the bottleneck to fast production shifts to manufacturing & iterating at a physical lab bench. the physical world is very messy and the difference between simulating electronics work & building it physically at our lab is huge. winners in this space will take design + test + ship + iterate + iterate again to speeds we've never seen before

TRISHALA avatar
TRISHALA@hitrishala·Nov 8

i’m telling you guys spy-as-a-service is a billion dollar market and this is just the start

TRISHALA avatar
TRISHALA@hitrishala·Nov 7

btw we rebranded 💸 something epic coming soon