←
Letta855 posts
Letta avatar

Letta

@Letta_AI

Building machines that learn.

◷ Joined August 2024▧ 321 Photos and videos
Letta avatar
Letta@Letta_AI·Oct 21

Introducing FrontierMemory: a new way to evaluate agent memory through downstream task performance. While existing memory benchmarks focus on recall (saturated via agentic search), FrontierMemory focuses on how agents learn from experience.

Post attachment
Letta avatar
Letta@Letta_AI·Nov 16

FrontierMemory scores memory by how much it helps a task agent, and we run both Opus 5.5 and GPT-6 Sol. Qwen4 Max's 73.3% comes with Sol, where its memories beat Opus 5.5's (73.3% vs 64.4%). With Opus 5.5 as the task agent, Opus's own memories still come out ahead (73.3% vs 68.9%).

Post attachment
Letta avatar
Letta@Letta_AI·Nov 16

Congrats to @Alibaba_Qwen on Qwen4 Max! It ties Opus 5.5 and Fable 5.2 for #1 on FrontierMemory at 73.3%, at about $54 per dream, under a fifth of Opus 5.5's cost.

Post attachment
Letta avatar
Letta@Letta_AI·Nov 13

Congrats to @GoogleDeepMind on Gemini 4 Carbon and Hydrogen! Carbon is 4th on FrontierMemory at 71.1%, a big jump over Gemini 4 Argon (64.4%), at about $94 per dream, a third of what Opus 5.5 costs. Hydrogen scores 64.4%, matching Argon, at about $100 per dream.

Post attachment
Letta avatar
Letta@Letta_AI·Nov 12

Inkling 2 Small's weights are now on Hugging Face under Apache 2.0. Congrats @thinkymachines and @tinkerapi. It's still the top US open-weights model on FrontierMemory: 60.0%, about $19 per dream.

Inkling 2 Small from @thinkymachines scores 60.0% on FrontierMemory, the highest of any US open-weights model, at about $19 per (American) dream.

Post attachment
Letta avatar
Letta@Letta_AI·Nov 11

Among the small models, Haiku 5.5 performs the best, nearing Opus 5.5 quality but at under a third of the cost per dream. GPT-6 Luna struggles to generate good memories, trailing at the bottom. Ember sits in-between the two, while being significantly cheaper than Luna.

Post attachment
Letta avatar
Letta@Letta_AI·Nov 11

Small models can dream too. We just added Meta’s new Muse Ember model to the FrontierMemory leaderboard, which measures an agent’s ability to self-improve through dreaming. Muse Ember performs similarly to Muse Wonder at a fraction of the cost.

Post attachment
Letta avatar
Letta@Letta_AI·Nov 10

Claude Fable 5.2 ties Opus for #1 on FrontierMemory. The quality of Fable 5.2's generated memories beat Opus 5.5 at every lower effort level. It's pricier to dream with, costing ~$600 per dream compared to ~$300 for Opus, 2x the cost.

Post attachment
Jun Jin avatar
⟳ Letta retweeted
Jun Jin@chris_jun_jin·Nov 7

Interesting work from @Letta_AI. I like the direction of FrontierMemory. Instead of testing whether an agent remembers something, they ask whether the memory actually helps the agent perform better on future tasks. This article also shows how difficult it is for models to extract reliable, actionable knowledge from past experience. My guess (not their disclosed architecture) is that Letta is moving from an LLM-based memory swarm toward ===>> a specialized memory learning model, possibly trained with downstream task rewards. Meta RL here can help! I see this as another approach to building an Agent Companion Model, similar to MemT. But I think there is a big challenge if they want to make this a commercial product. As the agent covers more and more tasks, and accumulates more experience, how can they keep inference fast and cheap? If each new task requires more memory retrieval, processing and reasoning, the cost may keep growing with experience. Moving memory processing offline helps, but doesn’t fully resolve the scaling problem. I think online plasticity could be the key here. But how to do this correctly? I don’t think anyone has a convincing solution yet. This is what I described as the Agent-CM trilemma: Task Breadth × Online Plasticity × Runtime Efficiency. trueintrinsics.app/writing/a…

Letta @Letta_AI Nov 3 Article Understanding how different models learn from agent experience We recently released FrontierMemory, a benchmark that tests whether agents get better at tasks by learning from past trajectories. On the leaderboard, memory written by Opus 5.5 led to a 73.3%

Will Knight avatar
⟳ Letta retweeted
Will Knight@willknight·Nov 7

Earlier this year, i wrote about startups including @Letta_AI and @LangChain exploring new ways for AI models to consolidate memories through something reminiscent of dreaming. Anthropic now says that Claude does something similar wired.com/story/sleeptime-co…

Memory is what allows Claude to learn from experience. In a new Engineering Blog post, we share how “dreaming” helps Claude organize what it’s learned, correct its mistakes, and find what it needs more easily the next time. anthropic.com/research/compo…

Sarah Wooders avatar
⟳ Letta retweeted
Sarah Wooders@sarahwooders·Nov 7

Very cool blog post from Anthropic that talks about how Claude uses sleep-time compute (from research @Letta_AI) to power learning from experience. Sleep-time compute is the most powerful form of agent learning today, but many unsolved problems remain!

Post attachment

Memory is what allows Claude to learn from experience. In a new Engineering Blog post, we share how “dreaming” helps Claude organize what it’s learned, correct its mistakes, and find what it needs more easily the next time. anthropic.com/research/compo…