←
Alex Morehead (何聪)560 posts
Alex Morehead (何聪) avatar

Alex Morehead (何聪)

@MoreheadAlex

Hopper Postdoctoral Fellow @BerkeleyLab. Prev: #MachineLearning & #CompBio PhD @Mizzou; Research Intern @ Profluent & Absci. #DeepLearning & #GenerativeModels.

◷ Joined December 2012▧ 49 Photos and videos
Alex Morehead (何聪) avatar
Alex Morehead (何聪)@MoreheadAlex·Oct 10

🧬 If you’d like to contribute to the next generation of open-source atomistic generative models in chemistry and biology, please reach out! (11/11)

Alex Morehead (何聪) avatar
Alex Morehead (何聪)@MoreheadAlex·Oct 10

Analyzing representations, scaling behavior, and generation quality, our results show that joint pretraining across molecular+materials data improves generative modeling and representation transfer, pointing toward general-purpose generative models for atomistic systems. (7/11)

Post attachment
Alex Morehead (何聪) avatar
Alex Morehead (何聪)@MoreheadAlex·Oct 10

Furthermore, we demonstrate that conditioning on atomic forces enables control over the degree of geometric relaxation in generated structures. (6/11)

Post attachment
Alex Morehead (何聪) avatar
Alex Morehead (何聪)@MoreheadAlex·Oct 10

Zatom-2 establishes strong performance on QM9, GEOM-Drugs, MP20, OMat24, and OMol25. When transferring to as few as 2,000 protein training examples, Zatom-2 generates designable backbones and extrapolates strongly to sequence lengths beyond the training distribution. (5/11)

Post attachment
Alex Morehead (何聪) avatar
Alex Morehead (何聪)@MoreheadAlex·Oct 10

This multitask pretraining leverages the rich geometric information in quantum-chemical calculations to shape the model's internal representations. (4/11)

Post attachment
Alex Morehead (何聪) avatar
Alex Morehead (何聪)@MoreheadAlex·Oct 10

Uniquely, Zatom-2 is pretrained on the large-scale OMol25 and OMat24 datasets using a multitask objective that combines flow matching generation and structure prediction with auxiliary machine learning interatomic potential (MLIP) tasks predicting energies and forces. (3/11)

Alex Morehead (何聪) avatar
Alex Morehead (何聪)@MoreheadAlex·Oct 10

Atomistic machine learning models typically target specific domains, such as small molecules, crystals, or proteins. Here, we introduce a unified atomistic generative model across these domains thanks to Zatom-2’s new atom1+atom14 tokenization scheme for chem+bio data. (2/11)

Minjia Zhang avatar
⟳ Alex Morehead (何聪) retweeted
Minjia Zhang@_Minjia_Zhang_·Jun 22

🧬 MegaFold: Efficient AlphaFold-3 Training Across NVIDIA and AMD GPUs MegaFold will be presented at ISC High Performance 2026! 🗺️Research Paper Session: ML Systems and Frameworks 🗓️ Wednesday, June 24 · 1:00–2:00 PM (CEST) 📍Hall E, 2nd Floor, Hamburg, Germany Come find out how we train AlphaFold-3 on 3.36× longer sequences, up to 1.73× faster, on both NVIDIA and AMD GPUs. Using MegaFold, we can successfully train on H200s, MI250 and MI325X. 🚀 Think training AlphaFold-3 (AF3) is just like training an LLM? Think again. Despite having only ~500M parameters, AF3 is up to two orders of magnitude slower to train than a size-equivalent LLM and OOMs on sequences as short as 590 residues. Why? One operator: 3D attention over 2D pairwise representations. It makes compute and memory scale cubically with sequence length, blowing up activation memory, fragmenting execution into tens of thousans of tiny kernels per iteration, and starving the GPU while the CPU grinds through evolutionary-context preprocessing. In our latest work, MegaFold, we diagnose exactly why AF3 training is slow and build an end-to-end system that fixes it across the entire stack. The results: ✅ 3.36× longer trainable sequences on 32 GPUs ✅ Up to 1.73× faster per-iteration training (1.62× on AMD) ✅ 23% lower peak memory ✅ Runs on both NVIDIA H200, AMD MI250 & AMD MI325x, fully cross-platform 🔧 Four key innovations: - EvoFlash-3D: a cross-platform Triton kernel that computes 3D attention without ever materializing the cubic attention map (cutting EvoAttention memory by 4.24×). - EvoSP-3D: a communication-efficient sharding strategy built for alternating-axis 2D pairwise representations - EvoFusion: a fused operator stack that tames AF3's kernel fragmentation - EvoPipe: a determinism-aware host-device pipeline that eliminates preprocessing stalls And it's verified: MegaFold matches baseline training loss step-for-step, so you get all the speed and memory savings with zero impact on convergence. Huge thanks to our students Hoa La and Ahan Gupta for working hard on this project, and to our collaborators Jianlin Cheng and Alex Morehead for their valuable contributions, feedback, and support throughout the work! MegaFold is fully open source. 🎉 💻 Code: github.com/Supercomputing-Sy… 🔗 Project page: supercomputing-system-ai-lab… 📄 Paper: arxiv.org/pdf/2506.20686 📝 Blog: supercomputing-system-ai-lab… See you at ISC, Hall E! 👋