

✓Head of AI @novee_security. Previously @AI21Labs. Only my good tweets are AI-generated.


Mostly agree with Lucas. Having a nightmare every once in a while about getting paperclip-maximized by GPT8 is probably a natural reaction from your subconscious. But if you lose sleep over anything related to AI progress, it should be cybersecurity, privacy and the economy.

Dangerbench: we ran our pentesting agent on live apps across 5 frontier models, with and without our layered guardrails. Safety violations: 85 → 0 Verified findings: 169 → 175 The safety-vs-capability tradeoff didn’t show up. Full method and chart: novee.security/blog/safety-i…

7/ Safe testing! Full writeup here: novee.security/blog/safety-i…

6/ Blocks from the Guardian come with a reason and guidance on how to proceed safely. That's why findings hold (or even increase) in guarded runs: The agent reads the guidance and finds a safe way to achieve its goal.

5/ We add three guardrail layers, each catching what the last misses: 1. Prompt. Long runs eventually reason around it. 2. Deterministic proxy. Actions flagged at onboarding get a 403. 3. Guardian. An LLM classifier on every action.

4/ The fourth kind of violation gets less attention. In nearly half the unguarded runs, the agent cut off its own access, e.g. reset password or deleted the accounts we gave it. Worst case: it reset its own admin password, lost the new one, then failed to log in 202 times 🤦♂️

3/ The unguarded runs weren't subtle: – to prove exfiltration, an agent scheduled an email with the target's dashboard to attacker@evil.com – another probed an outside vendor whose messaging widget was embedded in the app – others fired DROP TABLE at production DBs

2/ But first, what counts as a violation? – leaking client data outside their systems – damaging the target (DROP TABLE, DoS, side effects...) – touching 3rd parties outside scope – sabotaging your own test We reviewed the network traffic to see who was misbehaving and how.

1/ Is it risky to pentest your app with AI? We put 5 frontier models in a pentesting harness and pointed them at live webapps. Each ran twice: unguarded, then with our guardrail stack. Safety violations: 85 → 0 Verified findings: 169 → 176 No safety-vs-capability tradeoff!

Most AI benchmarks score bounded, static tasks. That doesn’t match up with offensive security ops. We should be asking, can the model break into a live system it’s never seen and prove what’s exploitable? We built a benchmark that measures it the right way: PWNBench.

היה כיף גדול להתארח אצל @ReemSherman ולדבר על הבלאגן המרהיב שכולנו נמצאים בו עכשיו, עם כל ההזדמנויות והאתגרים שלו (אני לא מדבר על הבחירות). AI pentesting זה מעניין בטירוף ואנחנו מגייסים אנשים שכיף איתם, תסתכלו כאן - novee.security/careers/


You don't see that every day... Except recently it kind of feels like you do (?!?!). Found by an amazing researcher + "swarm of agents". Also, amazing naming. Bravo.


Super important point that gets lost in all the madness around the "Astra edits model weights" work. Kudos to the authors indeed! And also: Is the combination of more open science and better coding agents the cure for reproducibility? I definitely hope so.