
you running a bunch of subscription accounts or paying API prices?

Encode AI for Science Fellow (Pillar VC / Imperial College London) Protein Design, PhD in machine learning for structural biology at UCL

you running a bunch of subscription accounts or paying API prices?

Link to paper: biorxiv.org/content/10.64898… Made possible by ESMFold2 model from @biohub and the excellent Mosaic software package from escalante.bio/

Our new preprint showing an interesting result we discovered while working on hallucination-based protein design: for binder design using ESMFold2 we found improvements from dropping softmax and replacing it with projected gradient descent. @dmmiller597


10k (Potts model) designed sequences for 3 proteins with labels from growth-based assay. 16-38% of sequences functional even at low sequence identity. Makes for a great benchmark of zero-shot fitness prediction methods in their ability to rank diverse sequences.

Sorry everyone, that was me trying to align two PETase sequences.
Breaking News: Anthropic said it halted potential plots by scientists who used its A.I. models to do research that could have helped develop biological weapons. nyti.ms/3V3epAE

1 more thing, because I don't want to be taken out-of-context by the AI haters: I use Claude every day and I remember what it was like trying to get open-source software to run in the pre-agent era: it was hell. So I'm eternally grateful for how much these tools speed up science

Great experiment and a great dataset: it shows that Claude can competently use the open-source tools that practitioners would use in a binder design campaign. but most of the magic is in the tools themselves and telling it which tools to use and how (the prompt)
Designing a binder is an easier process than designing a drug, but it’s a useful proxy. The typical success rate in the field today is between 10% and 15%. Between 22% and 35% of Claude's designs bound successfully, depending on the setup. Some of its strongest designs bound several times more tightly than the best published de novo binder.

if I knew I was competing on behalf of Team Human against the AIs, I would have increased our compute budget by 50X to match the amount used by Claude. and optimised more for hits rather than aiming for an informative benchmark set. Blog post coming soon on those 15PGDH results

I was particularly interested in this data because I also ran a binder-design experiment against 15-PGDH, benchmarking some of the same design tools that Claude used: we got 1 hit out of 100, while Claude Mythos got 1/30 with a stronger affinity. I will say...

more data like this helps us improve computational methods for binder design and, more generally, for modelling biophysical interactions. These methods have made incredible progress in the last few years and will be (already are) part of the toolkit for designing new drugs

as usual on x dot com there are hot takes on both sides: x.com/MartinShkreli/status/2… While it's true that these designed binders probably aren't directly useful for drug design, it's still impressive that AI agents can orchestrate these complicated pipelines. and...
this is not impressive work!!! affinities are quite low for peptidics. you can take business end of any mab & cut everything, too. these aren't useful probe molecules because notice none of them are intracellular! if i needed an extracellular probe, that's what a mab is for!!!

Once you read the prompt, it removes a lot of the degrees-of-freedom left for Claude to choose; it's mostly orchestrating the tools, re-allocating compute based on the ranking scores and making some design-specific choices like epitope and PDB structure

The prompt is an interesting document to read: it gives the exact formula for ranking candidate designs (including which models) and names seven structure-design tools that MUST each contribute ≥50 backbones per target before anything reaches the ranker

looks to me like the prompt is written by an expert who really knows the current state-of-the-art and the common failure modes when assigning Claude to this task. I learned a lot from reading it. I assume there was some iteration to get the prompt to cover all the failure modes

you can see that a tonne (16,500 words) of expert knowledge is passed to Claude explicitly, which tells it how to do the task: huggingface.co/datasets/Anth… It's still amazing that a single prompt and 24 hours get results this good. Regarding the prompt I will say...