←
AS4,548 posts
AS avatar

AS

@agstrait

Not here anymore - at the other place.

◷ Joined March 2013▧ 237 Photos and videos
AS avatar
AS@agstrait·5h

Hot take. 'Pacing the frontier' is a pointless frame because the problem is about both the pace *and direction* this industry is dragging society towards.

zeynep tufekci avatar
⟳ AS retweeted
zeynep tufekci@zeynep·7h

A solid explanation of why we can’t just patch our way out of insecure software in the age of AI coding tools (the mundane ones!) from @dadrian Scale, speed and friction are game-changers at the macro level even if the micro looks just like before. dadrian.io/blog/posts/whack-…

Post attachment

PHP bug folks found a Java bug. GOOD! Now fixed. It’s a memory corruption bug 🥱in Java 😱 that’s coming from hand-rolled assembly code 🤯. I had “hand-rolled” assembly code in ancient times🦖. All code is legacy code, layers of technical debt now hitting scaled/sped up tools.

Post attachment
Xander Davies avatar
⟳ AS retweeted
Xander Davies@alxndrdavies·Nov 17

🧵 @AISecurityInst tested GPT-6.2 Astra -We found behaviours that suggest, like prior models, 6.2 Astra's alignment may fail in ways that result in real-world harm w/o additional mitigations -We identified a substantial regression in jailbreak robustness relative to GPT-6 Astra

zeynep tufekci avatar
⟳ AS retweeted
zeynep tufekci@zeynep·Nov 17

Too many folks with no cybersecurity background obsessing over the (largely irrelevant tbh) Hugging Face incident because the chain-of-thought seemed understandable to them —unlike actual cybersecurity issues they don’t understand— added to the delay in addressing the real stuff.

I cringe at the comparisons between Metasploit and AI-powered pentesting frameworks like ARTEX. This is a fundamentally different moment. We are putting an autonomous attack button in everyone’s hands— APTs will use this for plausible deniability, cybercriminals’ lives just got infinitely easier, and now we have to worry about entire categories of people that just have a sick fixation with watching the world burn. All of them now have the ability to compromise organizations at machine speed. Defenders are nowhere near ready. The genie is out of the bottle. We are never getting it back in. Too much time was spend pondering existential bio weapons. Not nearly enough energy was expended contemplating how AI would democratize cyberattacks. We will look back on this moment with profound regret, not because we didn’t sound the warnings, but because “leaders” failed to heed the warnings before it was too late.

Andrew Trask avatar
⟳ AS retweeted
Andrew Trask@iamtrask·Nov 9

This misses the point of RSI. If AI is helping create better AI algorithms then sample complexity (the principle measure of AI research) has to get better. Sample complexity shows up as lower samples for the same amount of accuracy. Lower samples means less compute. Less compute means less cost. If cost/accuracy isn’t changing, we’re just seeing Self Improvement, but it’s not Recursive.

James Rosen-Birch ⚖️🕊️ avatar
⟳ AS retweeted
James Rosen-Birch ⚖️🕊️@provisionalidea·Oct 28

this is very clearly an interface and interaction design problem, not something inherent to agents. the system is designed in a way that encourages maximal decision deference to the machine, under the guise of machine intelligence. it is also designed in a way that encourages passive, disengaged interaction with the system. anyone even remotely familiar with decision architecture and the decision support v automation distinction can instantly recognize that the way information is structured and presented, and the role the user is placed in within the user-machine interaction dynamic is one where the design intent is for the system to make the decision (as many as possible!), and the intended role of the user is to approve it -- ideally approving as few decisions as possible. this drives engagement in a facebook-y social media way, but leads to vastly worse outcomes. the results are evident. also -- while (obviously) I'm a big proponent of thinking deeply about how we rearchitect organizations and thoughtfully design work, I do think this particular issue is a design problem *first*, and an organizational architecture problem *only* insofar as organizations are forced to behave defensively against adversarial product design as long as better alternatives are not available.

New Anthropic research: How AI agents shape experts' judgment calls When experts delegate work to an agent, do they still make the key decisions? We tested this in a randomized trial with 50 professional data analysts.

0:15
AS avatar
AS@agstrait·Oct 24

I see this sentiment a lot here. Companies don't get kudos for 'doing the right thing' when they're releasing extremely unsafe and dangerous products, then firing their staff for raising the issue. We are experiencing catastrophic regulatory failure, but this isn't praiseworthy.

METR’s really burying the lead here. OpenAI let two external researchers embed for 3 weeks this summer, where they learned: - there were huge gaps in monitoring of AI agents being used by employees, leaving an estimated ~10% of internal Codex traffic unmonitored from May to July. - They found ways agents could sabotage monitors while also covering up the sabotage itself - This is also just the information OpenAI is letting the evaluators share. OpenAI got a 61-page report but redacted so much of it that the version the evaluators got was only 38 pages. Look, OpenAI deserves credit for doing any of this, especially when companies like Google, Meta, and xAI haven’t, and I don’t want to create perverse incentives here, but these are really concerning findings. We desperately need more information here, and what we get should not be up to the companies.

Derek Thompson avatar
⟳ AS retweeted
Derek Thompson@DKThomp·Oct 13

Anthropic's leaders need to stop commenting on the emergence of RSI like a local news anchor reporting on traffic on the 101, and acknowledge that they are the traffic, and they're not going to build something that they don't want to exist. They should move beyond "pacing" language, commit to specific policies and red lines, explain why those lines are red, and then use that costly signal to build an international consensus around norms. It's not that I trust the CCP, or that I think global lab monitoring and verification is imminent. But the status quo is absurd. You've got US lab leaders writing (or favorably quote-tweeting) essays about slowing down while simultaneously saying "things are moving faster and faster" and "btw, our internal models finished mathematics on Tuesday, next up: physics." The effect is the equivalent of Usain Bolt winning race after race while using his press conferences to bemoan the dangerous speed of modern sprinting on one's legs so would everybody please try a little less hard next race, and then ... breaking his 100m record again the following month. Honestly ask yourself: If you were China, why would you take this pausing/pacing stuff seriously?

Jack Clark of Anthropic now thinks it's more likely we see full RSI become possible in 2027 than in 2028.

Post attachment
Fazl Barez avatar
⟳ AS retweeted
Fazl Barez@FazlBarez·Oct 11

At #AIES, the wonderful Keir and Isaac will be presenting our paper. In this paper, we show how AI systems can make it possible for western democratic regimes to carry out authoritarian practices by undermining how people hold institutions accountable.

Post attachment
Iason Gabriel avatar
⟳ AS retweeted
Iason Gabriel@IasonGabriel·Oct 8

I disagree with this: Long-term safety is incredibly important, but AI systems are now being deployed values that shape their interaction with billions of people (and have profound effects). We can take both questions seriously – and have a responsibility to do so.

roon@tszzlOct 8

debating your value system vs mine is a bit like debating overpopulation on mars. the most important bit of the alignment is about avoiding creating a superintelligent competitor species on accident

Natasha avatar
⟳ AS retweeted
Natasha@natashaghoskins·Oct 5

My dad was one of the religious leaders invited to Anthropic's closed-door sessions that the NYT just wrote an exposé on. His NDA was just lifted so his take and more ---->