
We’re releasing all 57 intermediate checkpoints from Marin “Mochi” 535B-A23B pretraining, including 10 from long-context extension. 18.2T tokens, about 61 TB of weights. huggingface.co/open-athena/M… Blog: openathena.ai/blog/marin-moc…

Faculty at UKY. Views my own, not of my employer(s). Work: #BioNLP, #NLProc, medical informatics, machine learning, LLMs, AI & fairness, health+socialdata

We’re releasing all 57 intermediate checkpoints from Marin “Mochi” 535B-A23B pretraining, including 10 from long-context extension. 18.2T tokens, about 61 TB of weights. huggingface.co/open-athena/M… Blog: openathena.ai/blog/marin-moc…

New paper w/ Yihe Dong, @DimitrisPapail & @ShivamGarg91462: Perturbing Inputs Improves Pretraining Distillation. In pretraining distillation, a small student LM learns to match a larger teacher’s next-token predictions on the training text. We change one thing: for the first part of training, both models read a perturbed version of the text (for example, with noise added to the token inputs), and the student matches the teacher’s predictions on that. Then we switch back to clean text. With the same steps and token budget, this beats standard distillation, and it also works well in data-constrained, multi-epoch settings. 🧵

The National Library of Medicine’s SPARK Challenges are here! There are two National Institutes of Health prize competitions advancing AI-enabled discovery across PubMed/PMC (35M+ citations) & dbGaP with total prizes up to $500,000: nlm.nih.gov/news/spark_chall… #DigitalHealth

Not able to attend #AMIA2026 this year. Folks attending, please post any cool papers or results you liked. Looking forward to year in review slides too.

We are like 1% of the US population, how are we replacing Christians again. And this is happening in Seattle 🤦♂️
HAPPENING NOW: Absolutely chaotic scene as conservative activists Jake Lang and Thomas Allen storm a Diwali Hindu celebration at the Seattle Center Armory. They held signs that read, "Deport All Dot Heads" and "Stop Hindu Replacement of Christians." I just spoke to both men who say they are concerned about American culture being overrun by false idol worshippers and H1-B fraudsters. Lang and Allen say they will be targeting other locations in the city this weekend. They have yet to announce details of their plans due to possible retaliation by Antifa and other far-left activists. Stay tuned. @JakeLang|@Tommy4Trump420

Very excited to introduce Paperclip Datasets! Curate your own datasets from millions of papers in minutes. Just describe the dataset you want to build! As example, we release the largest dataset of 20k half-life human measurements curated from 2400+ drugs💊 🖇️ Try it: paperclip.gxl.ai/datasets 📝 Blog: gxl.ai/blog/introducing-pape… 📊 Half-life dataset: huggingface.co/datasets/gxl-…

FAFO to all the racists who terrorized Indian people in TX 😇

Young'uns, tomorrow is the day 🙏
Young people, don't ignore midterm elections. Policies have long terms consequences for u & ur voice needs to be heard. Ur voting rate is already low during presidential elections & I am scared how awful it would be during midterms. Please vote on 11/3 (early voting is easier)

This is my own experience too. I don’t see hallucinated citations anymore.
2/n Take fabricated citations. A 2023 paper found that 55% of ChatGPT's references were invented. Using the same literature-review prompts, we measured about 1% for Claude Sonnet 5.5 and 0% for GPT-6.1 Sol. This once-familiar failure has become much rarer in newer models.

Wow! Is this also called "distillation"?
We asked GPT-6 Astra to edit Qwen3.5-0.8B weights to improve math. It directly changed only 0.016% of weights, fixing 0.8B's reasoning loops & improving math up to 6x and even instruction following 3x, without gradients or training. We call this Weight-Space Programming: frontier LLMs can now natively read/write neural network weights! 🧵 Blog: rikiyat.github.io/blog/weigh…

We're releasing one of the largest & most detailed look at how Americans use ChatGPT complementing reports from OpenAI, giving a much more deeper view into ChatGPT use Explore our interactive tools & analysis: stateoftheprompt.net Let us know if you want to see any analysis!
STATE OF THE PROMPT: Our Rutgers CSS Lab team analyzed a year of real ChatGPT conversations from 70K+ US panelists. See the new interactive tools, plus research reports on health, politics & more. stateoftheprompt.net/ With @gvrkiran @shagunjhaver @RutgersCommInfo @RutgersU

What's more weird is, I got this email ten days ago, and even today no such information is available in Commons. Everything is blank (no SRO, no PO, no study section). I called eRA customer service yesterday, and they said it's an automated email and I should check later :-)
You know what would reduce administrative burden, @NIH @NIH_CSR ? Instead of emailing a cryptic "your grant has been assigned to study section, log into eRA commons to find out which one" message you could just put the study section in the email.

Can an EHR model trained at one hospital read another hospital's records? Excited to introduce our new paper, "Discrete Neural Interlingua for Ontology-Agnostic EHR Modeling," accepted at NeurIPS 2026. See you in Sydney 🦘🏖️! Paper: arxiv.org/abs/2610.27047 Code: github.com/mahmoodlab/MEDIAT… Model: huggingface.co/MahmoodLab/ME… Video: See👇 The same clinical concept can be an ICD code, a SNOMED term, a local identifier, or free text, and standard EHR tokenizers treat each one as an unrelated symbol. A model trained on MIMIC-III with a closed vocabulary can use only 13% of eICU events. It isn't only a cross-hospital problem. For example, inside one hospital, "acute kidney injury" can be a billing code, a flowsheet identifier, and free text in a note, and the vocabulary shifts again with every coding transition and EHR migration. The redundancy is there before a model ever leaves the building. MEDIATOR is a discrete, ontology-agnostic tokenizer. Any clinical concept written as text maps to one of 100,000 shared tokens, so "acute kidney failure" (ICD-9), "AKI" (a local code), and "acute kidney injury" (free text) become the same token. It is trained on cross-ontology synonyms from UMLS and guided by LLM-enriched descriptions, code co-occurrence, and a medical relation graph. It is a drop-in token interface for standard EHR Transformers: no manual code mapping, and new code systems can be tokenized without retraining. Main results: • In-domain: +4.0 AUPRC points (8% relative) over SOTA, across 20 MIMIC-III tasks. Merging within-dataset redundancy pays off before you transfer anything. • Zero-shot transfer: on eICU with no vocabulary alignment, MEDIATOR uses all 3.58 million events and improves AUPRC over MedTok by 8.6 points (29% relative) across 14 tasks, up to +21.2 on subarachnoid hemorrhage. Restricted to the same inputs as the closed-vocabulary baselines, it is still the best tokenizer. • Transferable attribution: one shared token space means attributions compare directly across hospitals. For cirrhosis mortality, token importance is rank-correlated between MIMIC-III and eICU at Spearman ρ = 0.92. Tokenizing a dataset's entire vocabulary is a one-time step of about 70 seconds on one GPU. Code and weights are public, and the co-occurrence teacher can be rebuilt from public data with near-identical results. Congradulations to Caiwei Tian Tong Ding and everyone else. #NeurIPS2026 #ClinicalInformatics #EHR #MachineLearning

My issue not that they are using ChatGPT, it's fine, they are shooting their shot although my website says I cannot respond to prospective student emails. But their emails lately contain hallucinations attributing to me papers that I did not author. It's hurting their case now.
my question is, if you think I'm dumb enough to not realize your personalized email was tailored by ChatGPT, do you really want me to be your PhD advisor?

My Student Researcher project at @Google is out! 🔥 We introduce On-Policy Self-Training: Data-Free Self-Improvement in Large Language Models, showing that self-generated privileged information can effectively drive LLM self-improvement. 📄 Paper: arxiv.org/abs/2610.32770

🐭MICE to Meet You: From Re-ranking Only to End-to-End Retrieval ⚡️ Our #EMNLP2026 paper showed that MICE pushes the Cross-Encoder Pareto frontier. We take it a step further: turning theoretical pre-computation gains into an operational, end-to-end retrieval architecture 🧵