←
michael abbott3,240 posts
michael abbott avatar

michael abbott

@mabb0tt

building

◷ Joined March 2009▧ 15 Photos and videos
Ron Conway avatar
⟳ michael abbott retweeted
Ron Conway@RonConway·Nov 11

As the swirl continues (which I believe should be taken seriously), let’s pause for a moment and remember what Jensen has contributed to our industry with his determination and vision, which laid the foundation that we all work off of today. Jensen’s statements have shown grace and dignity to his family and the industry. He surely understands the impact. Now we should be patient and wait for investigations by the board of directors and any government authorities to conclude.

Larry Dial avatar
⟳ michael abbott retweeted
Larry Dial@classiclarryd·Nov 9

Sharing the thought process behind the architecture for Marin 535B-A23B, which finished pretraining on 18T tokens last week: openathena.ai/blog/marin-535… Earlier this year the Marin project had 6 full-time employees at Open Athena, a mix of AI veterans and people newer to the space, like me. I learned a lot from the team as an IC. Since then we got our B200 GPU stack running in a month, scaled up 300x on GPU to a 535 billion parameter model, publicly forecasted the run, live-streamed the entire thing, and made it through pretraining slightly better than predicted with zero instabilities. In the blog I call out specific contributions on architecture- but the broader gratitude is to the ecosystem. 2 years ago this would have been inconceivable on a first large scale attempt. Now it’s possible, in large part due to published work from Llama, DeepSeek, Qwen, Gemma, Olmo, Cohere, Nemotron, Arcee, Inkling, Mistral, Microsoft AI, Kimi, Muse, academia, and many others. The recipes are out there. A small team with big dreams (and some funding) can train a big model. (And if you have doubts on that, you can copy our fully open recipe from data to code). Of course, training a model does not produce a binary outcome. I am proud of how far we’ve come and optimistic about how much more there is to explore! The team has grown recently and will be taking the model through SFT and RL soon, sharing the findings along the way.

Post attachmentPost attachment
Percy Liang avatar
⟳ michael abbott retweeted
Percy Liang@percyliang·Nov 10

Why was Marin 535B-A23B the size it was? Why 8 of 384 experts? Why GatedNorm, sconv, XSA? Why linear learning rate? All of these factors were driven by tradeoffs between timelines, compute efficiency, MoE router stability, etc. Read about the rationale of each design decision:

Sharing the thought process behind the architecture for Marin 535B-A23B, which finished pretraining on 18T tokens last week: openathena.ai/blog/marin-535… Earlier this year the Marin project had 6 full-time employees at Open Athena, a mix of AI veterans and people newer to the space, like me. I learned a lot from the team as an IC. Since then we got our B200 GPU stack running in a month, scaled up 300x on GPU to a 535 billion parameter model, publicly forecasted the run, live-streamed the entire thing, and made it through pretraining slightly better than predicted with zero instabilities. In the blog I call out specific contributions on architecture- but the broader gratitude is to the ecosystem. 2 years ago this would have been inconceivable on a first large scale attempt. Now it’s possible, in large part due to published work from Llama, DeepSeek, Qwen, Gemma, Olmo, Cohere, Nemotron, Arcee, Inkling, Mistral, Microsoft AI, Kimi, Muse, academia, and many others. The recipes are out there. A small team with big dreams (and some funding) can train a big model. (And if you have doubts on that, you can copy our fully open recipe from data to code). Of course, training a model does not produce a binary outcome. I am proud of how far we’ve come and optimistic about how much more there is to explore! The team has grown recently and will be taking the model through SFT and RL soon, sharing the findings along the way.

Post attachmentPost attachment
Anjney Midha avatar
⟳ michael abbott retweeted
Anjney Midha@AnjneyMidha·Nov 9

earlier this year, @adamblake and i sat down with @NeginOliver @AMD and leading US scientists @periodiclabs to align on a plan to defend our frontier ecosystem from attacks we are facing from adversarial nations @mavriaX was born out of that plan excited to share more soon

@AnjneyMidha is the only person I know who might actually think bigger than we do. I'll always be grateful that he believed in us early ($30m day 1 check!). Excited to keep building together.

Sacred Heart Prep Athletics avatar
⟳ michael abbott retweeted
Sacred Heart Prep Athletics@SHPAthletics·Nov 7

Congratulations to Coach @dennis_omalley, the 2026 CCS Girls Flag Football Honor Coach! A pioneer in the sport, he led SHP to a 2nd place finish in the @wbal_ccs and the #4 seed in the Open Division playoffs. Grateful for his leadership and dedication to our student-athletes!

Post attachmentPost attachmentPost attachment
elie avatar
⟳ michael abbott retweeted
elie@eliebakouch·Nov 5

marin is the most exciting and ambitious open LLM training project. very excited to see this (big!) hero run finish pretraining, huge congrats to the amazing team working on this!! ⛵

Post attachment

It’s done! Marin’s Mochi 535B-A23B finished its main pretraining phase this morning. 18T tokens! Long context this weekend, SFT over the next week or so. Hoping to release checkpoints next week. Some EU paperwork first. mumtracker.exe.xyz/hero-run-…

Post attachment
David Hall avatar
⟳ michael abbott retweeted
David Hall@dlwh·Nov 5

It’s done! Marin’s Mochi 535B-A23B finished its main pretraining phase this morning. 18T tokens! Long context this weekend, SFT over the next week or so. Hoping to release checkpoints next week. Some EU paperwork first. mumtracker.exe.xyz/hero-run-…

Post attachment
Percy Liang avatar
⟳ michael abbott retweeted
Percy Liang@percyliang·Nov 4

Marin 535B-A23B now on 65K contexts... Counting down...

Post attachment
Alex Shan avatar
⟳ michael abbott retweeted
Alex Shan@alexshander03·Oct 27

Today, @JudgmentLabs opens for general access. We believe long running agent systems will generate more exhaust data than the world can currently process. Hidden within agent traces are rich records of how to make these systems better: what users ask for, when agents stumble, and how agents interact with the external world. Judgment is the most performant monitoring and improvement layer for long-horizon agents. We’re excited to see what you build!

2:09
Tyler Fong avatar
⟳ michael abbott retweeted
Tyler Fong@tyler_fong_·Oct 20

Today I’m joining @Natl_Compute I’ve spent the last four years working on GPU access, first as a founding engineer at Brev and then at NVIDIA after they acquired us in 2024. When I started, on-demand GPU capacity at the hyperscalers was effectively gone. If you were an independent researcher, a two-person startup, or a grad student with a grant and a deadline, you refreshed console pages and got nothing, or hammered the API only to get “insufficient capacity” errors back. I vividly remember hard-coding a filter at Brev to hide certain GCP instances because they almost never provisioned. Around 2023, a wave of new public clouds were coming online, and many of them had on-demand capacity. That gap is a big part of why Brev worked: we gave researchers a simple way to find GPUs across dozens of providers and start building. For a while, aggregation was enough. Then in 2025, I watched that capacity dry up too. Access to compute has never been worse. Existing supply is already under contract, and new capacity is reserved before it ever comes online. I won’t rehash why. There are a hundred good explanations out there covering debt financing, anchor tenants, backstops, lead times and so on. What matters more are the downstream effects. Long-term, single-tenant leases hold back scaling for everyone who couldn’t secure funding in time, and the industry’s momentum shouldn’t hinge on the handful of players who locked up access early. Today small startups, labs, and university researchers don’t even get a chance to compete. National Compute is solving this with a foundational layer we call “the Grid”. It’s an interconnected network of multi-silicon capacity composed of multiple providers and spans multiple sites that will be pooled for our members. The speed at which this team has been able to scale in such a small fraction of time is remarkable, and I’m excited to push that scale even higher for the public benefit.

hundreds of MI355x and B300 nodes now live on nationalcompute.com subsidized for all .edu and .govs it’s time to scale the frontier ecosystem for the public benefit

Post attachment
Will Held avatar
⟳ michael abbott retweeted
Will Held@WilliamBarrHeld·Oct 15

We're working on post-training our Marin MoE models. Many people ask us about where the environment data will come from. @dlwh wrote a bit why that's a great question and about how we're thinking about trying to tackle it.

I've been thinking about where the data for the next generation of open models will come from. The internet gave us a remarkable commons for pretraining. Posttraining is different: it consumes tasks, not documents. And that commons doesn't exist. Not yet. openathena.ai/blog/there-is-…