←
Jude Wells375 posts
Jude Wells avatar

Jude Wells

@_judewells

Encode AI for Science Fellow (Pillar VC / Imperial College London) Protein Design, PhD in machine learning for structural biology at UCL

◷ Joined March 2009▧ 99 Photos and videos
Jude Wells avatar
Jude Wells@_judewells·Oct 26

Our new preprint showing an interesting result we discovered while working on hallucination-based protein design: for binder design using ESMFold2 we found improvements from dropping softmax and replacing it with projected gradient descent. @dmmiller597

Post attachment
Jude Wells avatar
Jude Wells@_judewells·Sep 17

10k (Potts model) designed sequences for 3 proteins with labels from growth-based assay. 16-38% of sequences functional even at low sequence identity. Makes for a great benchmark of zero-shot fitness prediction methods in their ability to rank diverse sequences.

Post attachment
Jude Wells avatar
Jude Wells@_judewells·Aug 19

1 more thing, because I don't want to be taken out-of-context by the AI haters: I use Claude every day and I remember what it was like trying to get open-source software to run in the pre-agent era: it was hell. So I'm eternally grateful for how much these tools speed up science

Jude Wells avatar
⟳ Jude Wells retweeted
Jude Wells@_judewells·Aug 19

Great experiment and a great dataset: it shows that Claude can competently use the open-source tools that practitioners would use in a binder design campaign. but most of the magic is in the tools themselves and telling it which tools to use and how (the prompt)

Designing a binder is an easier process than designing a drug, but it’s a useful proxy. The typical success rate in the field today is between 10% and 15%. Between 22% and 35% of Claude's designs bound successfully, depending on the setup. Some of its strongest designs bound several times more tightly than the best published de novo binder.

Post attachment
Jude Wells avatar
Jude Wells@_judewells·Aug 19

if I knew I was competing on behalf of Team Human against the AIs, I would have increased our compute budget by 50X to match the amount used by Claude. and optimised more for hits rather than aiming for an informative benchmark set. Blog post coming soon on those 15PGDH results

Jude Wells avatar
Jude Wells@_judewells·Aug 19

I was particularly interested in this data because I also ran a binder-design experiment against 15-PGDH, benchmarking some of the same design tools that Claude used: we got 1 hit out of 100, while Claude Mythos got 1/30 with a stronger affinity. I will say...

Jude Wells avatar
Jude Wells@_judewells·Aug 19

more data like this helps us improve computational methods for binder design and, more generally, for modelling biophysical interactions. These methods have made incredible progress in the last few years and will be (already are) part of the toolkit for designing new drugs

Jude Wells avatar
Jude Wells@_judewells·Aug 19

as usual on x dot com there are hot takes on both sides: x.com/MartinShkreli/status/2… While it's true that these designed binders probably aren't directly useful for drug design, it's still impressive that AI agents can orchestrate these complicated pipelines. and...

this is not impressive work!!! affinities are quite low for peptidics. you can take business end of any mab & cut everything, too. these aren't useful probe molecules because notice none of them are intracellular! if i needed an extracellular probe, that's what a mab is for!!!

Jude Wells avatar
Jude Wells@_judewells·Aug 19

Once you read the prompt, it removes a lot of the degrees-of-freedom left for Claude to choose; it's mostly orchestrating the tools, re-allocating compute based on the ranking scores and making some design-specific choices like epitope and PDB structure

Jude Wells avatar
Jude Wells@_judewells·Aug 19

The prompt is an interesting document to read: it gives the exact formula for ranking candidate designs (including which models) and names seven structure-design tools that MUST each contribute ≥50 backbones per target before anything reaches the ranker

Jude Wells avatar
Jude Wells@_judewells·Aug 19

looks to me like the prompt is written by an expert who really knows the current state-of-the-art and the common failure modes when assigning Claude to this task. I learned a lot from reading it. I assume there was some iteration to get the prompt to cover all the failure modes

Jude Wells avatar
Jude Wells@_judewells·Aug 19

you can see that a tonne (16,500 words) of expert knowledge is passed to Claude explicitly, which tells it how to do the task: huggingface.co/datasets/Anth… It's still amazing that a single prompt and 24 hours get results this good. Regarding the prompt I will say...