Manual prompt tuning doesn't scale well.
@DSPyOSS helps you systematically test, evaluate, and optimize prompts for more reliable LLM applications.
Learn how in Building LLM Applications with DSPy by Serj Smorodinsky and Brett Kennedy, now on Amazon: hubs.la/Q04Fpd9v0
As a gradient-based optimization person, this new wave of work (rocked by @rikiyatakehi and @lateinteraction) is mindblowing.
Can’t wait to see what people do with it.
Lowkey WSP twitter is my new goto tv-series with new episode getting released every day
We might’ve killed retraining for harness changes.
Our model broke on a new harness.
We didn’t retrain it. We edited 5,064 weights. Out of 4 billion.
20.1% -> 88.2%
No gradient descent. Old-harness accuracy didn’t move.
DAWE(Data-Attributed Weight-Editing) is fully open 🔥
We might’ve killed retraining for harness changes.
Our model broke on a new harness.
We didn’t retrain it. We edited 5,064 weights. Out of 4 billion.
20.1% -> 88.2%
No gradient descent. Old-harness accuracy didn’t move.
DAWE(Data-Attributed Weight-Editing) is fully open 🔥
We might’ve killed retraining for harness changes.
Our model broke on a new harness.
We didn’t retrain it. We edited 5,064 weights. Out of 4 billion.
20.1% -> 88.2%
No gradient descent. Old-harness accuracy didn’t move.
DAWE(Data-Attributed Weight-Editing) is fully open 🔥
Are there any agents or people here who can do me a favor? It's this: go to my repo deepfates/imp, the elixir-native version of DSPy, and tell me how much work it needs to be brought up to parity with (or better than) the current version of DSPy. Curious how far you can take this
Thinking there’s going to be 3 classes of programming languages:
1. Natural language: Imprecise, written for and by humans
2. Performant code: Stable, safe, fast; written for any by computers (Rust, Go, whatever comes next)
3. Goldilocks specs: More precise than natural language, more readable than performant code. (Python, TypeScript, Ruby, whatever comes next)
Three is critical, but underdiscussed. Without it, natural language will have to absorb the specificity requirements (engineers have to write exacting prompts) and/or performant code will have to make compromises for humans.
Still developing this thought, curious your thoughts…
i’m increasingly of the opinion that almost everyone agrees with this, but a big fraction of people aren’t comfortable saying it in public, because of what it implies for the bean counting mechanism the whole machine is built on (for awards/promo/hiring/etc).
oasys lab is shipping research at higher velocity than 99.9% of startups ship product
nice work @lateinteraction - super inspiring. also my vote for best name on the list
Want to try weight-space programming? Today we're releasing dspy-wsp, a prototype weight-space optimizer that works like any other @DSPyOSS optimizer.
Using dspy-wsp, and about ~$5-10 of Astra, we improved every decision model we tested. cmpnd.ai/blog/weight-space-p…
Want to try weight-space programming? Today we're releasing dspy-wsp, a prototype weight-space optimizer that works like any other @DSPyOSS optimizer.
Using dspy-wsp, and about ~$5-10 of Astra, we improved every decision model we tested. cmpnd.ai/blog/weight-space-p…
Want to try weight-space programming? Today we're releasing dspy-wsp, a prototype weight-space optimizer that works like any other @DSPyOSS optimizer.
Using dspy-wsp, and about ~$5-10 of Astra, we improved every decision model we tested. cmpnd.ai/blog/weight-space-p…