←
Yangjun Ruan254 posts
Yangjun Ruan avatar

Yangjun Ruan

@YangjunR

Compressing

◷ Joined February 2021▧ 24 Photos and videos
Yangjun Ruan avatar
Yangjun Ruan@YangjunR·26 Mar 2025

New paper on synthetic pretraining! We show LMs can synthesize their own thoughts for more data-efficient pretraining, bootstrapping their capabilities on limited, task-agnostic data. We call this new paradigm “reasoning to learn”. arxiv.org/abs/2503.18866 Here’s how it works🧵

Post attachment
Yangjun Ruan avatar
Yangjun Ruan@YangjunR·18h

Many ppl are talking about personalized AI that know all your context and understand you, but there are deep technical challenges around infra & privacy. @kenziyuliu really made it! Congrats on the release, and excited to see how it will power the new generation of personal AI!!

Introducing Second, and a $10M seed led by @a16z. Second builds a model of you that you own, and your agents can use as an API. Truly personal software needs a deep understanding of you: not just your emails and messages, but how you work, what motivates you, your people, priorities, style, goals, and intent. This requires a new kind of personal AI infrastructure. One that continuously aggregates your context, distills it into durable understanding with a ton of compute, and then serves it securely, ubiquitously, and up-to-the-minute. One that builds your *portable user model*. Second is that infrastructure. It installs seamlessly on your Mac, and your context just comes out. Live APIs of your priorities, to-dos, wikis, writing samples, preferences. Or what you did minutes ago. Or context packs of anything, dynamically assembled in seconds. Second is streaming-state from local to a private cloud, which means anything you connect to--whether Claude, Dot, Muse, Grok Bot, Instinct, or iMessage--shares the same, growing, continuous understanding of you. Your portable user model is as powerful as it is sensitive. We founded @FirstPersonLabs on the principle that it should belong to you. This commitment means taking on hard systems and security problems and making sovereignty an invariant of Second from the beginning. We've barely scratched the surface of what portable user models can unlock. Deep understanding of us, served at machine speeds, could drive the next leap in personal software, human-computer interaction, large-scale coordination of humans and agents, and more! We're tremendously grateful to be supported by @a16z (@martin_casado, @YODELAGARZA), Mozilla Ventures, @JeffDean, Ram Shriram, Jeetu Patel (@jpatel41), @percyliang, J. Alex Halderman (@jhalderm), @sanmikoyejo, and many others. Our vision is for a future where every person is more powerful, because they have the ultimate affordance: personal computers that fully understands them on their own terms. If this excites you, we're hiring and come build with us!

1:23
Thinking Machines avatar
⟳ Yangjun Ruan retweeted
Thinking Machines@thinkymachines·Jul 15

Today, we are introducing Inkling. Inkling reasons efficiently across text, image, and audio modalities. We are making the full weights available. thinkingmachines.ai/news/int… Available today for fine-tuning on Tinker. Play with it in the Inkling Playground. 🧵

Tatsunori Hashimoto avatar
⟳ Yangjun Ruan retweeted
Tatsunori Hashimoto@tatsu_hashimoto·May 21

Some new results I found surprising that I’m tweeting for Chris (who isnt on here). With enough compute, the best data filter for LMs (on DCLM) might be no filter. Why? Large models can tolerate a surprising amount of nominally 'low quality' data, and can sometimes even benefit.

Post attachment
Thinking Machines avatar
⟳ Yangjun Ruan retweeted
Thinking Machines@thinkymachines·May 11

People talk, listen, watch, think, and collaborate at the same time, in real time. We've designed an AI that works with people the same way. We share our approach, early results, and a quick look at our model in action. thinkingmachines.ai/blog/int…

2:15
John Yang avatar
⟳ Yangjun Ruan retweeted
John Yang@jyangballin·May 5

How much of SQLite, FFmpeg, PHP compiler can LMs code from scratch? Given just an executable and no starter code or internet access. Introducing ProgramBench: 200 rigorous, whole-repo generation tasks where models design, build, and ship a working program end to end. 🧵

Post attachment
David Duvenaud avatar
⟳ Yangjun Ruan retweeted
David Duvenaud@DavidDuvenaud·Apr 27

Announcing Talkie: a new, open-weight historical LLM! We trained and finetuned a 13B model on a newly-curated dataset of only pre-1930 data. Try it below! with @AlecRad and @status_effects 🧵

7:58
Ken Liu avatar
⟳ Yangjun Ruan retweeted
Ken Liu@kenziyuliu·Apr 15

Sharing a super simple, user-owned memory module we've been playing around: nanomem The basic idea is to treat memory as a pure intelligence problem: ingestion, structuring, and (selective) retrieval are all just LLM calls & agent loops on a on-device markdown file tree. Each file lists a set of facts w/ metadata (timestamp, confidence, source, etc.); no embeddings/RAG/training of any kind. For example: - `nanomem add <fact>` starts an agent loop to walk the tree, read relevant files, and edit. - `nanomem retrieve <query>` walks the tree and returns a single summary string (possibly assembled from many subtrees) related to the query. What’s nice about this approach is that the memory system is, by construction: 1. partitionable (human/agents can easily separate `hobbies/snowboard.md` from `tax/residency.md` for data minimization + relevance) 2. portable and user-owned (it’s just text files) 3. interpretable (you know exactly what’s written and you can manually edit) 4. forward-compatible (future models can read memory files just the same, and memory quality/speed improves as models get better) 5. modularized (you can optimize ingestion/retrieval/compaction prompts separately) Privacy & utility. I'm most excited about the ability to partition + selectively disclose memory at inference-time. Selective disclosure helps with both privacy (principle of least privilege & “need-to-know”) and utility (as too much context for a query can harm answer quality). Composability. An inference-time memory module means: (1) you can run such a module with confidential inference (LLMs on TEEs) for provable privacy, and (2) you can selectively disclose context over unlinkable inference of remote models (demo below). We built nanomem as part of the Open Anonymity project (openanoymity.ai), but it’s meant to be a standalone module for humans and agents (e.g., you can write a SKILL for using the CLI tool). Still polishing the rough edges! - GitHub (MIT): github.com/OpenAnonymity/nan… - Blog: openanonymity.ai/blog/nanome… - Beta implementation in chat client soon: chat.openanonymity.ai Work done with amazing project co-leads @amelia_kuang @cocozxu @erikchi !!

0:30
Konwoo Kim avatar
⟳ Yangjun Ruan retweeted
Konwoo Kim@konwookim·Mar 20

for data-constrained pre-training, synth data isn’t just benchmaxxing, it lowers loss on the real data distribution as we generate more tokens for even better scaling, treat synth gens as forming one long 𝗺𝗲𝗴𝗮𝗱𝗼𝗰: 1.8x data efficiency with larger gains under more compute

Post attachment
Christina Baek avatar
⟳ Yangjun Ruan retweeted
Christina Baek@_christinabaek·Mar 18

Models are typically specialized to new domains by finetuning on small, high-quality datasets. We find that repeating the same dataset 10–50× starting from pretraining leads to substantially better downstream performance, in some cases outperforming larger models. 🧵

Post attachment
Ken Liu avatar
⟳ Yangjun Ruan retweeted
Ken Liu@kenziyuliu·Feb 26

Can we build a blind, *unlinkable inference* layer where ChatGPT/Claude/Gemini can't tell which call came from which users, like a “VPN for AI inference”? Yes! Blog post below + we built it into open source infra/chat app and served >15k prompts at Stanford so far. How it helps with AI user privacy: # The AI user privacy problem If you ask AI to analyze your ChatGPT history today, it’s surprisingly easy to infer your demographics, health, immigration status, and political beliefs. Every prompt we send accumulates into an (identity-linked) profile that the AI lab controls completely and indefinitely. At a minimum this is a goldmine for ads (as we know now). A bigger issue is the concentration of power: AI labs can easily become (or asked to become) a Cambridge Analytica, whistleblow your immigration status, or work with health insurance to adjust your premium if they so choose. This is a uniquely worse problem than search engines because your average query is now more revealing (not just keywords), interactive, and intelligence is now cheap. Despite this, most of us still want these remote models; they’re just too good and convenient! (this is aka the "privacy paradox".) # Unlinkable inference as a user privacy architecture The idea of unlinkable inference is to add privacy while preserving access to the remote models controlled by someone else. A “privacy wrapper” or “VPN for AI inference”, so to speak. Concretely, it’s a blind inference middle layer that: (1) consists of decentralized proxies that anyone can operate; (2) blindly authenticates requests (via blind signatures / RFC9474,9578) so requests are provably sandboxed from each other and from user identity; (3) relays prompts over randomly chosen proxies that don’t see or log traffic (via client-side ephemeral keys or hosting in TEEs); and (4) the provider simply sees a mixed pool of anonymous prompts from the proxies. No state, pseudonyms, or linkable metadata. If you squint, an unlinkable inference layer is essentially a vendor for per-request, anonymous, ephemeral AI access credentials (for users or agents alike). It partitions your context so that user tracking is drastically harder. Obviously, unlinkability isn’t a silver bullet: the prompt itself still goes to the remote model and can leak privacy (so don't use our chat app for a therapy session!). It aims to combat *longitudinal tracking* as a major threat to user privacy, and its statistical power increases quickly by mixing more users and requests. Unlinkability can be applied at any granularity. For an AI chat app, you can unlinkably request a fresh ephemeral key for every session so tracking is virtually impossible. # The Open Anonymity Project We started this project with the belief that intelligence should be a truly public utility. Like water and electricity, providers should be compensated by usage, not who you are or what you do with it. We think unlinkable inference is a first step towards this “intelligence neutrality”. # Try it out! It’s quite practical - Chat app “oa-chat”: chat.openanonymity.ai/ (<20 seconds to get going) - Blog post that should be a fun read: openanonymity.ai/blog/unlink… - Project page: openanonymity.ai/ - GitHub: github.com/OpenAnonymity

Post attachment
CLS avatar
⟳ Yangjun Ruan retweeted
CLS@ChengleiSi·Jan 22

Can LLMs automate frontier LLM research, like pre-training and post-training? In our new paper, LLMs found post-training methods that beat GRPO (69.4% vs 48.0%), and pre-training recipes faster than nanoGPT (19.7 minutes vs 35.9 minutes). 1/

Post attachment
Yangjun Ruan avatar
Yangjun Ruan@YangjunR·Jan 23

I always think TTT as the best scientific setup for studying data efficiency in the limit - and here we have some signs of life that there are very data-efficiency learning paradigms

How to get AI to make discoveries on open scientific problems? Most methods just improve the prompt with more attempts. But the AI itself doesn't improve. With test-time training, AI can continue to learn on the problem it’s trying to solve: test-time-training.github.io…

Post attachment
Yangjun Ruan avatar
Yangjun Ruan@YangjunR·Jan 12

We've seen pretraining as such a powerful learning paradigm by compressing information in the context into weights - now we should start doing that at test time, too.

LLM memory is considered one of the hardest problems in AI. All we have today are endless hacks and workarounds. But the root solution has always been right in front of us. Next-token prediction is already an effective compressor. We don’t need a radical new architecture. The missing piece is to continue training the model at test-time, using context as training data. Our full release of End-to-End Test-Time Training (TTT-E2E) with @NVIDIAAI, @AsteraInstitute, and @StanfordAILab is now available. Blog: nvda.ws/4syfyMN Arxiv: arxiv.org/abs/2512.23675 This has been over a year in the making with @arnuvtandon and an incredible team.

Post attachment