←
Dan Padnos538 posts
Dan Padnos avatar✓

Dan Padnos ✓

@DanPadnos

Head of AI @novee_security. Previously @AI21Labs. Only my good tweets are AI-generated.

◷ Joined July 2019▧ 39 Photos and videos
Dan Padnos avatar
Dan Padnos@DanPadnos·Nov 16

הדבר היחיד שהולך להישרף זה הכפתור של הלייקים

1:07
Dan Padnos avatar
Dan Padnos@DanPadnos·Nov 15

Mostly agree with Lucas. Having a nightmare every once in a while about getting paperclip-maximized by GPT8 is probably a natural reaction from your subconscious. But if you lose sleep over anything related to AI progress, it should be cybersecurity, privacy and the economy.

Novee Security avatar
⟳ Dan Padnos retweeted
Novee Security@novee_security·Nov 15

Dangerbench: we ran our pentesting agent on live apps across 5 frontier models, with and without our layered guardrails. Safety violations: 85 → 0 Verified findings: 169 → 175 The safety-vs-capability tradeoff didn’t show up. Full method and chart: novee.security/blog/safety-i…

Post attachment
Dan Padnos avatar
Dan Padnos@DanPadnos·Nov 12

6/ Blocks from the Guardian come with a reason and guidance on how to proceed safely. That's why findings hold (or even increase) in guarded runs: The agent reads the guidance and finds a safe way to achieve its goal.

Post attachment
Dan Padnos avatar
Dan Padnos@DanPadnos·Nov 12

5/ We add three guardrail layers, each catching what the last misses: 1. Prompt. Long runs eventually reason around it. 2. Deterministic proxy. Actions flagged at onboarding get a 403. 3. Guardian. An LLM classifier on every action.

Post attachment
Dan Padnos avatar
Dan Padnos@DanPadnos·Nov 12

4/ The fourth kind of violation gets less attention. In nearly half the unguarded runs, the agent cut off its own access, e.g. reset password or deleted the accounts we gave it. Worst case: it reset its own admin password, lost the new one, then failed to log in 202 times 🤦‍♂️

Dan Padnos avatar
Dan Padnos@DanPadnos·Nov 12

3/ The unguarded runs weren't subtle: – to prove exfiltration, an agent scheduled an email with the target's dashboard to attacker@evil.com – another probed an outside vendor whose messaging widget was embedded in the app – others fired DROP TABLE at production DBs

Dan Padnos avatar
Dan Padnos@DanPadnos·Nov 12

2/ But first, what counts as a violation? – leaking client data outside their systems – damaging the target (DROP TABLE, DoS, side effects...) – touching 3rd parties outside scope – sabotaging your own test We reviewed the network traffic to see who was misbehaving and how.

Dan Padnos avatar
Dan Padnos@DanPadnos·Nov 12

1/ Is it risky to pentest your app with AI? We put 5 frontier models in a pentesting harness and pointed them at live webapps. Each ran twice: unguarded, then with our guardrail stack. Safety violations: 85 → 0 Verified findings: 169 → 176 No safety-vs-capability tradeoff!

Post attachment
Novee Security avatar
⟳ Dan Padnos retweeted
Novee Security@novee_security·Nov 11

Most AI benchmarks score bounded, static tasks. That doesn’t match up with offensive security ops. We should be asking, can the model break into a live system it’s never seen and prove what’s exploitable? We built a benchmark that measures it the right way: PWNBench.

Post attachment
Dan Padnos avatar
Dan Padnos@DanPadnos·Nov 7

היה כיף גדול להתארח אצל @ReemSherman ולדבר על הבלאגן המרהיב שכולנו נמצאים בו עכשיו, עם כל ההזדמנויות והאתגרים שלו (אני לא מדבר על הבחירות). AI pentesting זה מעניין בטירוף ואנחנו מגייסים אנשים שכיף איתם, תסתכלו כאן - novee.security/careers/

Post attachment
Dan Padnos avatar
Dan Padnos@DanPadnos·Nov 5

You don't see that every day... Except recently it kind of feels like you do (?!?!). Found by an amazing researcher + "swarm of agents". Also, amazing naming. Bravo.

Dan Padnos avatar
Dan Padnos@DanPadnos·Nov 3

Super important point that gets lost in all the madness around the "Astra edits model weights" work. Kudos to the authors indeed! And also: Is the combination of more open science and better coding agents the cure for reproducibility? I definitely hope so.