←
Chris Kelly1,171 posts
Chris Kelly avatar

Chris Kelly

@chrisck

Health at Microsoft AI. Previously @GoogleHealth, @GoogleDeepMind, @KingsImaging. Paediatric doctor at @EvelinaLondon. Created NeoMate.co.uk

◷ Joined March 2009▧ 162 Photos and videos
Chris Kelly avatar
Chris Kelly@chrisck·6h

Workflow if you’re interested! 1️⃣ Paper + lyrics outline → GPT Astra → iteration on song lyrics until happy 2️⃣ Lyrics → Suno (~30th time lucky!) → mp3 3️⃣ GitHub Copilot + Opus 5.5: brainstorming on video and style, passing in song, paper, lyrics → initial storyboard, and building the rendering engine - entirely in JavaScript 🤯 4️⃣ Character and style generation using a mix of MAI Image 2.6 and GPT Image 2.5 5️⃣ GitHub Copilot: ~20 rounds of build → multiple internal loops of agent feedback → detailed human feedback (me!), before starting the round again… The first version was really very bad, so it does require a lot of careful iteration and refinement… Code to make yours... github.com/chrisk1/one-more-… Science Paper: science.org/doi/10.1126/scie…

Chris Kelly avatar
Chris Kelly@chrisck·6h

Should all academic papers come with a music video? 😛 I made one for our recent Science paper on AI sequential diagnosis. Turn your sound on! 🎶 Built with GitHub Copilot over a couple of weeks as a personal experiment - it’s fun to see what’s possible... Workflow and code below - looking forward to seeing your future academic paper music videos... 🤓

2:17
Eric Horvitz avatar
⟳ Chris Kelly retweeted
Eric Horvitz@erichorvitz·Aug 4

High-stakes decisions with language models We introduce 'decision-analytic steering' in this study: arxiv.org/pdf/2608.01361 In real-world decisions, different types of errors rarely have equal consequences. So, a system making recommendations under uncertainty must do more than predict what is likely to be true. It must also reflect the relative costs and benefits of acting or failing to act. As an example, in emergency medical triage, missing a patient who needs immediate care is often far more costly than sending someone to the emergency department who ultimately did not need to go. It's critically important to consider that asymmetry. We set out to investigate opportunities for decision-analytic steering by generating ROC curves for families of model. These sample curves showing the true positive and false positive rate for two models on a set of emergency medicine triage cases.

Post attachment
elie avatar
⟳ Chris Kelly retweeted
elie@eliebakouch·Jun 3

microsoft MAI tech report is a gold mine, one of the most transparent for a model at this scale. this model uses zero synthetic data or distillation from previous models. this means reasoning, agentic behavior, tool use are all learned fully during post-training with no cold start. bold choice that makes it harder and requires more iterations to reach sota, but you get FULL control over your model series and it proves they are serious about being a frontier lab. the tech report is insanely detailed and precise about numbers. to give an example, they give the exact MFU across all the iterations of the model, with the exact changes etc. they also share the full scaling ladder recipe, to my knowledge this is the first time i've seen this in a tech report at this scale let's look at all of this in this likely very long thread 🧵

Post attachment

Super excited to announce seven new world-class MAI models today. They represent what we consider a new era in AI designed to keep you in control and on the frontier. First is our text foundation model, MAI-Thinking-1, exceptionally strong on reasoning and SWE tasks. - It’s a 35B active parameter MoE with a 256K context window. Independent human raters on Surge prefer it for overall quality in blind side-by-sides versus Sonnet 4.6, and it’s achieved 97% on AIME 2025, the key measure of its general-purpose reasoning abilities. - It's at 53% on SWE Bench Pro, placing it right alongside Opus 4.6 on one of the toughest coding benchmarks. - And since we co-designed our models with our own silicon, MAI-Thinking-1 is optimized on our MAIA 200 chip. Benchmarking head-to-head against the GB200, we see 30% better performance per dollar as well as a 1.4x performance-per-watt gain when running our MAI models on the MAIA 200 end-to-end. Next is MAI-Image-2.5 and its Flash variant. Two super strong models now at #2 on the leaderboards, surpassing the score of Nano Banana 2 on image editing. Last for now is MAI-Code-1-Flash, our new inference efficient coding model, especially tuned for VS Code and GitHub Copilot CLI. - Code-1-Flash achieves 51% on SWE Bench Pro, despite having just 5B parameters, putting it closer to Haiku in size but cheaper in cost. All of this is the foundation for Microsoft Frontier Tuning. It lets you customize our models to create custom, company-specific agents that only you control. You can make our model, your model. Your data. Your agents. Your moat. Early adopters are already seeing a difference. When we tuned our models for McKinsey’s tasks, MAI delivered the highest win rate, outperforming GPT-5.5 on quality, while being 10x lower on cost. Also really excited to be collaborating with the amazing team at Mayo Clinic to jointly train a new frontier AI model for healthcare. Our announcements today mark another milestone on the road to humanist superintelligence. You can learn more and about our other new models in our latest blog: microsoft.ai/news/building-a…

Post attachment
Chris Kelly avatar
Chris Kelly@chrisck·Mar 12

We've all long imagined a health companion that truly knows you - not a chatbot that gives generic advice, but something that accumulates understanding of your health over time, and gets smarter the more you use it. Today we're launching Copilot Health: a secure, separate space within Copilot where you can bring together your health records from 50,000+ US provider orgs, data from 50+ wearable devices, lab results, and health history all into one place. Copilot Health then applies medical intelligence to make sense of it all, giving you real agency over your own health. Your data, understood in context, patterns surfaced that you might not spot alone, alongside the confidence to walk into your doctor's appointment more informed, prepared, and empowered. We are at a unique inflection point in history - AI is reaching extraordinary capabilities in medicine, while health data is finally being mobilised...and almost everyone has a phone in their pocket (or knows someone with one). We're starting in the US, but ultimately want to bring medical expertise to the billions of people who have never had access to it. Read more here (microsoft.ai/news/introducin…) and you can sign up to the waitlist to help shape what comes next!

Chris Kelly avatar
Chris Kelly@chrisck·Mar 10

Really excited 🥳 to share two breast cancer AI papers from my time at Google, published jointly in Nature Cancer today! We set out in 2021 to answer a question that matters to millions of women: can AI safely improve breast cancer screening in the NHS? Five years, five organisations, and 125,000+ women’s scans later, here's what we found. 1️⃣ Our first paper (nature.com/articles/s43018-0…) evaluated Google's mammography AI across five NHS screening services, with 39-month follow-up including interval + next-round cancers: → AI achieved superior sensitivity to human readers (54% vs 44%, P<0.001) with non-inferior specificity → 25% of future interval + next-round cancers detected = potential for earlier diagnosis → Reading time reduced by 32% while cancer detection increased by 18% → No systematic disparities across age, ethnicity, deprivation, or breast density → Prospective deployment at 12 sites confirmed feasibility but revealed distribution shift requiring recalibration - a critical lesson for implementation 2️⃣ Our second paper (nature.com/articles/s43018-0…) tackled what happens when AI becomes the second reader. When readers disagree today, a specialist panel "arbitrates". We studied 50,000 women's screens with 22 readers, with and without AI as the second reader: → End-to-end including arbitration, our AI-enabled arm was non-inferior to standard double reading (P<0.001) → Human reading workload reduced by 46% → AI flagged far more interval + next-round cancers before arbitration, but many were overruled, even when the AI correctly localised the cancer → Future: better explainability, prior image integration, reader training, and new pathways to maximise AI success (e.g. supplemental imaging for high risk normal cases) An editorial from Allan Hackshaw and Rosalind Given-Wilson (nature.com/articles/s43018-0…) covers this work really well - thank you! Conclusion: The AI works, and it can find cancers earlier. But how we integrate it into clinical workflows will determine whether that potential translates into better outcomes for women. This collaboration between @GoogleResearch, @imperialcollege, @RoyalSurrey, @stgeorgeshospital, St George's University Hospitals NHS Foundation Trust, and Imperial College Healthcare NHS Trust was funded by the NHS AI Award. We are deeply grateful to everyone involved. Thank you to @skourti_elena at Nature Cancer. Congratulations Lucy Warren, Marc Wilson, Jenny Venton, Ken Young, Mark Halling-Brown, Megumi Morigami, Lisanne Khoo, Deborah cunningham, Richard Sidebottom, Reddy Mamatha, Hema Purushothaman, Delara Khodabakhshi, Lesley Honeyfield, Amandeep Hujan, Tsvetina Stoycheva, Andy Joiner, Reena Chopra, Aminata Sy, Dominic Ward, Lin Yang, Rory Sayres, Daniel Golden, Namrata Malhotra, Rachita Mallya, Lihong Xi, Della Ogunleye, Charlotte Purdy, Alistair Mackenzie, Jane Chang, Jonathan Dixon, Elzbieta Gruzewska, Emma Lewis, Marcin Sieniek, Shawn Xu, @DrSusanThomas, @shravyas, @fjg28_fiona, @Ara_Darzi, Hutan Ashrafian 🎉

Post attachmentPost attachment
Nathan Benaich avatar
⟳ Chris Kelly retweeted
Nathan Benaich@nathanbenaich·9 Oct 2025

🪩The one and only @stateofai 2025 is live! 🪩 It’s been a monumental 12 months for AI. Our 8th annual report is the most comprehensive it's ever been, covering what you *need* to know about research, industry, politics, safety and our new usage data. My highlight reel:

25:19
Chris Kelly avatar
Chris Kelly@chrisck·30 Jun 2025

Takeaway: AI shouldn’t just know the answer to a single question; it must ask the right questions at the right time, adjust given new information, weigh up costs/invasiveness of investigations, and collaborate deeply like the best physicians. While just a game at this stage, SDBench + MAI-DxO feels like an important step towards that future.

Chris Kelly avatar
Chris Kelly@chrisck·30 Jun 2025

Takeaway: AI shouldn’t just know the answer to a single question; it must ask the right questions at the right time, adjust given new information, weigh up costs/invasiveness of investigations, and collaborate deeply like the best physicians. While just a game at this stage, SDBench + MAI-DxO feels like an important step towards that future.

Chris Kelly avatar
Chris Kelly@chrisck·30 Jun 2025

📊 Results: MAI-DxO + o3 hits 80% accuracy vs 20% for UK/US physicians (=4x better), at a 20% reduced diagnostic spend vs doctors (70 % vs vanilla o3). Plug it into OpenAI models, Gemini, Claude, Grok, DeepSeek, Llama - the orchestrator improves performance across the board.

Post attachment
Chris Kelly avatar
Chris Kelly@chrisck·30 Jun 2025

🤖 Introducing MAI-DxO: our model-agnostic orchestrator spinning up a role-based “panel” of virtual doctors with different perspectives and values, debating differentials and choosing only high-value tests. MAI-DxO triggers "chain of debate" for each step, helping improve performance.

Post attachment
Chris Kelly avatar
Chris Kelly@chrisck·30 Jun 2025

Start with a short ~two sentence abstract, then ask ANY follow-up question or order ANY test. The system returns real findings, or high-fidelity simulated responses, on demand. Two fundamental metrics: Did they get it right? And what did it cost?

Post attachment
Chris Kelly avatar
Chris Kelly@chrisck·30 Jun 2025

🔍 Our new "Sequential Diagnosis" Benchmark (SDBench) takes 304 real @NEJM cases and turns them into interactive diagnostic generative-AI-powered games that both human physicians and agents can play.

Post attachment
Chris Kelly avatar
Chris Kelly@chrisck·30 Jun 2025

New paper today! 🥳 How good is generative AI at diagnosis compared to human doctors? We introduce a novel, interactive medical benchmark (SDBench) for “sequential diagnosis”, and an orchestrator (MAI-DxO) that achieved over 4x higher diagnostic accuracy vs experienced physicians who played the benchmark. 🧵

Post attachment