←
Mayee Chen747 posts
Mayee Chen avatar

Mayee Chen

@MayeeChen

Research @EngramLab. previously CS PhD @StanfordAILab @HazyResearch, undergrad @princeton. Working on all things data! she/her 🎃

◷ Joined February 2020▧ 29 Photos and videos
Mayee Chen avatar
Mayee Chen@MayeeChen·Jun 23

I've spent my PhD figuring out how a model learns from data, digging through piles of Spanish math problems, fanfiction, and the depths of StackExchange. Engram opens a new dimension: how a model learns from you - your world, how it evolves, and the threads that tie it all together. Figuring out how to do this well is a wide-open research problem. Join us :)

Engram @EngramLab Jun 23 Article Introducing Engram: Scaling compute on your context We’re Engram. We’re building AI that learns from you and deeply understands your work. Today’s AI models don’t understand what you do. Not really. Everything models know comes from their training –

Together AI avatar
⟳ Mayee Chen retweeted
Together AI@togethercompute·Nov 16

Two weeks ago we announced Together Compiler: agents that tune inference to your model, chip, and traffic, for up to 2.6x more throughput. Today, we’re sharing the journey behind it: the experiments that challenged our assumptions of megakernels, the surprising strengths of Torch Compile, and the lessons that shaped our development of a next-generation system for inference optimization. Shout out to @dylan__lim @reyna_abhyankar @simran_s_arora and the team! Read more: together.ai/blog/road-to-tog…

Post attachment
Anthropic avatar
⟳ Mayee Chen retweeted
Anthropic@AnthropicAI·Nov 6

Memory is what allows Claude to learn from experience. In a new Engineering Blog post, we share how “dreaming” helps Claude organize what it’s learned, correct its mistakes, and find what it needs more easily the next time. anthropic.com/research/compo…

Neel Guha avatar
⟳ Mayee Chen retweeted
Neel Guha@NeelGuha·Nov 6

We released LegalBench in August 2023. In the three years since, it's become one of the more widely used benchmarks for legal AI. In the spirit of critical self-reflection, I wrote some thoughts about (1) what went well, (2) what I wish I'd done differently, and (3) what this all means for the future of legal AI research. I wanted to highlight a few specific reflections.

Post attachment
Jessy Lin avatar
⟳ Mayee Chen retweeted
Jessy Lin@realJessyLin·Nov 4

We've been doing some science on how training <> harness memory fit together, and consistently find they stack -- the strongest systems use all kinds of memory Excited to share some results on a clean and simple setting! Teaching models to use their notes with RL augments base models with memory, and taking gradients is actually pretty cheap I think the trend will be that a lot of harness/prompt eng today will matter less as we just let models learn how to operate with what they have at their disposal. This is especially relevant as text memory and raw context blurs together - as we've seen in codebases exploding with agent-generated content, learning how to juggle docs and skill.mds itself is becoming a skill :)

In our new blog, we share some new science investigating how RL complements text-based memory by letting models learn how to use their notes. It turns out that training can be cheap compared to building notes, and stays surprisingly robust even through millions of tokens of edits.

0:03
Julio Pereyra avatar
⟳ Mayee Chen retweeted
Julio Pereyra@ItsJulioPereyra·Nov 5

Great new work from the @EngramLab team! Co-optimizing not just the tools an agent uses but the context it operates in is a great direction for recurring knowledge bases which are common in legal. Their figure game also continues to be some of the best in the business!

In our new blog, we share some new science investigating how RL complements text-based memory by letting models learn how to use their notes. It turns out that training can be cheap compared to building notes, and stays surprisingly robust even through millions of tokens of edits.

0:03
Jack Morris avatar
⟳ Mayee Chen retweeted
Jack Morris@jxmnop·Nov 4

these days i'm quite excited about the prospect of models that understand more about our data than we do one barrier we've run into is that when models run for a long time, they take increasingly inscrutable notes; it turns out that small amounts of training can fix this also fun to share a glimpse into the kind of work we've been up to at Engram! expect more soon

In our new blog, we share some new science investigating how RL complements text-based memory by letting models learn how to use their notes. It turns out that training can be cheap compared to building notes, and stays surprisingly robust even through millions of tokens of edits.

0:03
Dan Biderman avatar
⟳ Mayee Chen retweeted
Dan Biderman@dan_biderman·Nov 4

There's an explosion of AI-generated context, which today's agents have to re-read in every session. For now, agents can mostly get by with text memory alone, but in 12 months this will be too slow and expensive. AI agents will have to learn a new way to navigate that context. Engram is building the memory layer that will make this possible, combining text-based memories with training.

In our new blog, we share some new science investigating how RL complements text-based memory by letting models learn how to use their notes. It turns out that training can be cheap compared to building notes, and stays surprisingly robust even through millions of tokens of edits.

0:03
Scott Linderman avatar
⟳ Mayee Chen retweeted
Scott Linderman@scott_linderman·Nov 4

Continual learning is about more than internalizing new knowledge — it’s about learning to use memory effectively. Excited to share some of our research at Engram showing how training complements text memory, making agents more intelligent, cost-effective, and robust.

In our new blog, we share some new science investigating how RL complements text-based memory by letting models learn how to use their notes. It turns out that training can be cheap compared to building notes, and stays surprisingly robust even through millions of tokens of edits.

0:03
Mayee Chen avatar
Mayee Chen@MayeeChen·Nov 4

Sharing a bit of what we've been up to on the research side! These days, agents mostly accumulate learnings through notes, wikis, and skill files. What if we trained them to use these notes better? The result I'm most excited about - training stacks on top of text memory systems as they evolve. Training once, even on an older version of the notes, still helps after 23M tokens of edits!

In our new blog, we share some new science investigating how RL complements text-based memory by letting models learn how to use their notes. It turns out that training can be cheap compared to building notes, and stays surprisingly robust even through millions of tokens of edits.

0:03
Engram avatar
⟳ Mayee Chen retweeted
Engram@EngramLab·Nov 4

In our new blog, we share some new science investigating how RL complements text-based memory by letting models learn how to use their notes. It turns out that training can be cheap compared to building notes, and stays surprisingly robust even through millions of tokens of edits.

0:03
Simran Arora avatar
⟳ Mayee Chen retweeted
Simran Arora@simran_s_arora·Nov 4

Introducing the Together Compiler: continual compilation for AI inference! I’m super proud to finally share this work from our research team! ✨ Today’s inference is “Polymorphic” or shape-shifting, it doesn’t sit still. On OpenRouter, 80% of traffic goes to models released in the last three months and the top ten most popular open models each use a different architecture. Our customers constantly change their usage behaviors. Together Compiler leverages agents to continually optimize our deployments immediately as these changes occur. Together Compiler research highlights include: 1. 252x better throughput: TKCC lets agents try 252x more optimization experiments per hour compared to prior agent inference optimization systems. 2. State-of-the-art TPS across InferenceX workloads we tried: TKCC outperforms the best InferenceX engine across all tested settings in just 24 hours & 2 nodes per compile. Including 1111 TPS on MiniMax M3.1 B200 and 1205 TPS on DeepSeek v4.1 Flash GB300 at batch size 1. NVIDIA, AMD, Trainium2. 3. Continuous recompilation: TKCC currently serves many trillions of production tokens per day and adapts to traffic shifts within just a few hours of recompilation. 4. Bitter lessons: TKCC’s design builds on 2+ years of building frameworks to accelerate AI Inference, starting from ThunderKittens and Megakernels. Compilers have always used abstractions and IRs, but we show that agentic compilers now reach higher performance ceilings without abstractions. From an incredible team @reyna_abhyankar @dylan__lim @realDanFu @togethercompute!!

Alexis Ross avatar
⟳ Mayee Chen retweeted
Alexis Ross@alexisjross·Oct 30

announcing &tutor, a tutor we trained against persimmon! there are many reasons I'm so excited about this. here are a few: 1. this is a case study into the differentiated bet that we're making at humans&: having good models of users (e.g. learners) allows us to train AI models that can help people over long time horizons (e.g. teach in a way that imparts new understanding) 2. education is a near and dear application close to my heart, and I've been wanting to work on something like this since starting my PhD 3. @314Bansal is the 🐐🐐🐐 and it was so much fun working together on this

Introducing Ampersand Tutor (&Tutor) - we trained a model to learn like people, then trained &Tutor to teach it On a widely-used tutor model eval, &Tutor performed best, showing how better models of people can help build AI for humans Available through our API - try it!