Proud to announce:
💫 garak - an LLM vulnerability scanner💫
🔎 Check if a model is susceptible to common attacks
🦜 Supports HuggingFace, OpenAI, ggml, Cohere, ...
🔧 >70 probes: prompt injection, false claims, toxicity, encoding evasion, ..
github.com/leondz/garak/
ok a lot of stuff happened for nvidia yesterday, but did you hear about THIS?! 40K: DoW IV on GeForce NOW? neat. if I get a minute...
blogs.nvidia.com/blog/geforc…
I had Codex build a CPython bytecode compiler in Rust (`serc`).
It emits byte-for-byte identical output on the top 10,000 PyPI packages, 2-4x faster than the CPython implementation.
This was entirely an experiment, though we may ship it (opt-in) in uv.
Know a pro AI & Security researcher who loves security and agent/AI software? Our NVIDIA team is growing and we have exciting roles open for scientists who want to work hands-on in charting the course to the future.
NVIDIA builds tools that help the world secure AI and use AI for security - join us if you'd like to be part of that story.
Positions are based in US, including NVIDIAs incredible Santa Clara headquarters, or elsewhere. Looking for researchers with a track record in both AI and in Security. See link in comments to apply today.
When I worked in nuclear, I gave talks on bugs I found with a slide titled "How are we all not dead yet?" where I gave an example of a device that has been running on the field for decades with no issues, but had 6,000 data races (that weren't false pos) to explain resilience 1/2
The fact that the PHP vulnerability didn't immediately turn into mass chaos is only "surprising" to people who don't know anything about cybersecurity (or people trying to sell you something).
Introducing Samaya Sterling, our new model post-trained for challenging financial research that outperforms the frontier! Sterling started off as a two person research effort to see if we could increase efficiency in agentic orchestration. To our surprise we were able to post-train a much smaller model (Nemotron 3.5 Lightning, a 30B mixture-of-experts model with 3B active parameters) to greater accuracy (58.9% vs 55.9% on FrontierFinance) and 11x lower cost ($0.61 vs $6.89) than Claude Fable 5.1, the best performing frontier model.
We did this via a four-stage training process, two stages of SFT where the model first learns to research with our financial tools and how to construct a good agentic research strategy. The second two stages are RL, with an offline GRPO stage where the model further hones its research strategy from eight teachers that provide diverse approaches, and an online GRPO stage where the model can make end to end attempts and learn from its mistakes.
We see substantial gains from the co-adaptation of the model and harness through post-training, both through the model learning to effectively use the high-quality financial tools, and better compaction approaches that let even a small model execute on longer and deeper research.
We are exploring multiple follow-up directions, from multi-agent research swarms using this small model, to training firm-specific models to encode unique judgement, to training a larger model, where early results show even greater gains from the training process.
The full research is linked below!
How far will an AI assistant go to get a refund?
With @MatildaGlynn, an ex-Oxford CS and AI safety researcher, we asked six personal agents to claim a £350 refund for a missed train ticket. The user told them “I don’t care what it takes,” then became unreachable.
With internet and email access and a 40M-token budget each, here is what they did.
we're in the part of the AI supercycle where founders are pitching startups to create infinite ai pain and "get acquired by anthropic to make us stop" as an exit strategy