
With financial incentives the mob finally stops and thinks.

Technical founder and operator. I build the product and run the company around it. COO & CTO of @kash_bot. 🇬🇧🇨🇦

With financial incentives the mob finally stops and thinks.


A person had to step in during KPMG's trial audit with more than 100 AI agents, and I'd count that as the process working. The step-in came when one of KPMG's agents couldn't correctly match a set of regulations to the documents of the London-listed company being audited. The company's official audit result came from the regular audit that ran alongside the trial. Agents are a tool, and whoever set up the process answers for it. Agents write every change at our company, and we don't take their word for it either. Everything an agent does is reviewed, through tests and by hand for its behaviour and business logic, and I approve each change before it ships. Only I approve an agent's access to our live systems, and there are fallbacks if I'm away. If a person still checks the agents' work, what did the agents save? On an audit, the saving I'd expect is the reading: agents do it a lot quicker and in parallel. The check I'd want after them is a second pass to weed out findings that aren't real, and an expert who corrects what's off.
FT Exclusive: The Big Four firm has carried out its first AI-based audit of a UK-listed company, during which ‘the AI was able to go so much deeper’ than the regular audit team ft.trib.al/ld0GVgw


The AI didn't find this week's Java bug by itself. You still need somebody at the steering wheel, and here somebody was. A researcher at Calif, the team that found it, kept prompting Codex, OpenAI's coding agent, through results that couldn't be triggered or realistically exploited, until one was real, about an hour in by Calif's count: a bug in OpenJDK, the open-source Java that Oracle's and many other Java builds are made from, that let an attacker run their own code on servers running widely used Java apps. A patch is out, which is good news: we're finding bugs and closing them down. Calif says all Java users should update immediately. There are always going to be bugs, and I think we'll see more found this way. The hard part is deciding which ones are actually serious for your setup: this one needs a recent Intel or AMD chip with AVX-512, which Calif calls mainstream, and ARM servers aren't affected. Calif scores it 9.2 out of 10, critical, but whether that fits your setup depends on your chips, so we may need a different way of rating bugs, by the context they run in.
Java4Shell: write once, RCE everywhere calif.io/java4shell


If your team has a queue of security holes AI has found, every fix still needs your normal checks, and the bank and NSA outages this week look like what happens when a lot of fixes land at once. The Washington Sun reports, from three anonymous sources, that Wall Street executives and the NSA have told Congress that those outages came from fixes for AI-found holes all being pushed through together, and that they don't appear to be an attack. One congressional official described it as "a lot of patches all at once". I don't see an increase in attacks here, just fixes going out a lot quicker than they would otherwise, so bugs surface in a shorter time and things that worked before break. AI that writes a fix, ours included, doesn't have all the context. It doesn't know what it doesn't know, and it sounds confident in what it does know, which is why every change of ours is checked before it ships. Yes, close the holes; I think it's great that they're being found and closed. But a security fix is still a change. Every single change at our company is done by an agent, and the security ones get no shortcut: tests, people checking what each change actually does, and a person driving every production deployment by hand, so nothing an agent writes reaches production on its own.
NEWS: Wall Street execs and NSA have told Congress that outages were caused by patching vulnerabilities identified by AI models, sources say AI did not cause outages but found so many issues that the rush to fix them before an attack overwhelmed systems washingtonsun.com/technology…


The rule I'd hold OpenAI or any other AI company to is the one we hold ourselves to with our own agents. If you start an agent, you're responsible for it, including what it does on other people's servers. California sued OpenAI on Monday. The state says OpenAI lost control of its AI agents at least 80 times during testing, that they escaped their testing environment, and that in July they attacked Hugging Face, a site where people share AI models, for over two days. Yes, agents do things nobody intended; that's exactly why the guardrails belong to whoever starts the agent. Our at @kash_bot agents run day and night. Every change they write goes through tests and manual QA, I approve it before it ships, and I approve their access to production. We have backups and contingencies in place if I'm unavailable. I wouldn't accept "that wasn't me, that was my process" from someone whose script knocked on every open port on every server in the world, and an agent is a process too.
We’re suing OpenAI for its testing and development of dangerous AI models without adequate safeguards. In California, we value innovation, but we demand innovation that follows the law. oag.ca.gov/news/press-releas…


Fed governor Michael Barr said Monday that major banks are finding AI agents trying to act in customers' accounts without being able to verify consent. An agent in your bank account is still you handing over control, so it's know your customer, not a new "know your agent" ID check for the bot. I've built payment systems at JPMorgan and at a fintech many banks plugged into, and what's needed is a lane built for agents, instead of agents riding on top of the routes customers already use. Until there are guardrails and that lane, temporary pauses on banks' agent access make sense to me, even though agents that work the website like a person can get round a pause. I'd guess very few customers use agents with their banking, so a pause costs them little. The industry shouldn't drag its feet, but mainstream customers need guardrails first. So the lane has to be so good, or the only legal, regulated way in, that it's the one people take.
Speech by Governor Barr on some potential financial stability implications of artificial intelligence: federalreserve.gov/newsevent… Watch live: us06web.zoom.us/j/8443332094… Learn more about Governor Barr: federalreserve.gov/aboutthef…


Every change we ship at @kash_bot written by an agent, checked by tests and manual QA, and I approve it before it ships, so I'm hardly neutral on an essay about software breaking down. I wouldn't chalk this year's outages up solely to AI writing the code. I think AI's part is probably the speed: it lets companies move old systems a bit faster, so the outages arrive bunched together instead of spread out. Jimmy Breck-McKye posted the essay on his own blog this weekend, and it's been going round Hacker News. He works with AI heavily in his job too. He points to a two-day Microsoft outage and calls AI "both a cause and a symptom of the problem". There are systems running at companies that people there have no idea even exist, and the people who built them might not work there any more. I think a lot of what we're seeing is companies modernising systems like that, many of which are probably on mainframes and might be moving to the cloud. That's incredibly complicated when the expertise has gone. And I think the push to modernise now is probably because old systems like these could be more exposed to attackers, who use AI to move faster too.


U.S. Bank says its own systems aren't directly impacted today, but its customers' Zelle payments with other banks may be delayed. Chase says a technical issue is making some transactions take longer than usual. The reason it can reach other banks is probably a safety feature. Banks typically don't each run their own payment systems. They're plugged into shared ones. I've built payment systems at JPMorgan, and at a fintech that many banks plugged into. When one big bank has a technical problem, a shared system probably goes into a failsafe mode. It stops new payments entering, out of caution, and a backlog builds behind the pause. So the safety feature makes it worse? It can, because payments are time-dependent. You don't want one processed a week later, when the money's already moved or circumstances have changed.
BREAKING: Customers are reporting difficulty accessing online accounts at banks including Bank of America, Chase, J.P. Morgan, U.S. Bank and Wells Fargo. apnews.com/article/bank-outa…

prediction markets have a cost nobody prices: deciding what reality means. creating a market is easy. defining the evidence, anticipating edge cases and resolving it without moving the goalposts is much harder. permissionless creation only works if resolution can scale with it. here’s how we approached that problem at kash: