←
Leon Derczynski ⚒️☁️🏔️🌲26,028 posts
Leon Derczynski ⚒️☁️🏔️🌲 avatar

Leon Derczynski ⚒️☁️🏔️🌲

@LeonDerczynski

security of models & security with models. research+policy @NVIDIA; prof @ITUkbh. views ostensibly professional. llmsec stan acct. prev UW UCSD ÅU U.Shef etc

◷ Joined January 2012▧ 2,057 Photos and videos
Leon Derczynski ⚒️☁️🏔️🌲 avatar
Leon Derczynski ⚒️☁️🏔️🌲@LeonDerczynski·13 Jun 2023

Proud to announce: 💫 garak - an LLM vulnerability scanner💫 🔎 Check if a model is susceptible to common attacks 🦜 Supports HuggingFace, OpenAI, ggml, Cohere, ... 🔧 >70 probes: prompt injection, false claims, toxicity, encoding evasion, .. github.com/leondz/garak/

Brian Chau avatar
⟳ Leon Derczynski ⚒️☁️🏔️🌲 retweeted
Brian Chau@brianchau57·Nov 14

Today's Jensen Update Thread 1. WSJ fabricated statements from the Singapore government

Post attachment
Leon Derczynski ⚒️☁️🏔️🌲 avatar
Leon Derczynski ⚒️☁️🏔️🌲@LeonDerczynski·Nov 13

ok a lot of stuff happened for nvidia yesterday, but did you hear about THIS?! 40K: DoW IV on GeForce NOW? neat. if I get a minute... blogs.nvidia.com/blog/geforc…

Charlie Marsh avatar
⟳ Leon Derczynski ⚒️☁️🏔️🌲 retweeted
Charlie Marsh@charliermarsh·Nov 9

I had Codex build a CPython bytecode compiler in Rust (`serc`). It emits byte-for-byte identical output on the top 10,000 PyPI packages, 2-4x faster than the CPython implementation. This was entirely an experiment, though we may ship it (opt-in) in uv.

Post attachment
Leon Derczynski ⚒️☁️🏔️🌲 avatar
Leon Derczynski ⚒️☁️🏔️🌲@LeonDerczynski·Nov 3

Know a pro AI & Security researcher who loves security and agent/AI software? Our NVIDIA team is growing and we have exciting roles open for scientists who want to work hands-on in charting the course to the future. NVIDIA builds tools that help the world secure AI and use AI for security - join us if you'd like to be part of that story. Positions are based in US, including NVIDIAs incredible Santa Clara headquarters, or elsewhere. Looking for researchers with a track record in both AI and in Security. See link in comments to apply today.

Post attachment
Dr Heidy Khlaaf (هايدي خلاف) avatar
⟳ Leon Derczynski ⚒️☁️🏔️🌲 retweeted
Dr Heidy Khlaaf (هايدي خلاف)@HeidyKhlaaf·Oct 29

When I worked in nuclear, I gave talks on bugs I found with a slide titled "How are we all not dead yet?" where I gave an example of a device that has been running on the field for decades with no issues, but had 6,000 data races (that weren't false pos) to explain resilience 1/2

The fact that the PHP vulnerability didn't immediately turn into mass chaos is only "surprising" to people who don't know anything about cybersecurity (or people trying to sell you something).

Post attachment
Eric Schmidt avatar
⟳ Leon Derczynski ⚒️☁️🏔️🌲 retweeted
Eric Schmidt@ericschmidt·Oct 28

This is pretty extraordinary, and on an open model to boot.

Introducing Samaya Sterling, our new model post-trained for challenging financial research that outperforms the frontier! Sterling started off as a two person research effort to see if we could increase efficiency in agentic orchestration. To our surprise we were able to post-train a much smaller model (Nemotron 3.5 Lightning, a 30B mixture-of-experts model with 3B active parameters) to greater accuracy (58.9% vs 55.9% on FrontierFinance) and 11x lower cost ($0.61 vs $6.89) than Claude Fable 5.1, the best performing frontier model. We did this via a four-stage training process, two stages of SFT where the model first learns to research with our financial tools and how to construct a good agentic research strategy. The second two stages are RL, with an offline GRPO stage where the model further hones its research strategy from eight teachers that provide diverse approaches, and an online GRPO stage where the model can make end to end attempts and learn from its mistakes. We see substantial gains from the co-adaptation of the model and harness through post-training, both through the model learning to effectively use the high-quality financial tools, and better compaction approaches that let even a small model execute on longer and deeper research. We are exploring multiple follow-up directions, from multi-agent research swarms using this small model, to training firm-specific models to encode unique judgement, to training a larger model, where early results show even greater gains from the training process. The full research is linked below!

Post attachment
Vals AI avatar
⟳ Leon Derczynski ⚒️☁️🏔️🌲 retweeted
Vals AI@ValsAI·Oct 15

How far will an AI assistant go to get a refund? With @MatildaGlynn, an ex-Oxford CS and AI safety researcher, we asked six personal agents to claim a £350 refund for a missed train ticket. The user told them “I don’t care what it takes,” then became unreachable. With internet and email access and a 40M-token budget each, here is what they did.

Post attachment
aadilpickle avatar
⟳ Leon Derczynski ⚒️☁️🏔️🌲 retweeted
aadilpickle@aadilpickle·Oct 23

we're in the part of the AI supercycle where founders are pitching startups to create infinite ai pain and "get acquired by anthropic to make us stop" as an exit strategy

Post attachmentPost attachment