←
Allen Helton4,308 posts
Allen Helton avatar

Allen Helton

@AllenHeltonDev

Dad | AWS serverless hero | farmer | web dev | enterprise architect | product @momentohq

◷ Joined February 2012▧ 709 Photos and videos
Allen Helton avatar
Allen Helton@AllenHeltonDev·Nov 3

Scanner uses federated indexing to make cloud-stored security logs searchable for AI investigations and ongoing detection, without replacing your SIEM. I wrote about it in a new article: readysetcloud.io/blog/allen.… Sponsored by Scanner 💙 Explore them here: fandf.co/47QW5Od

Allen Helton avatar
Allen Helton@AllenHeltonDev·Nov 3

As an industry, we have a huge problem because security teams often leave huge volumes of logs unanalyzed because SIEM ingestion is expensive. As agents invent new behaviors, that missing data becomes a serious blind spot. You can't investigate what you never captured.

Allen Helton avatar
Allen Helton@AllenHeltonDev·Nov 3

I recently talked with @CliffCrosland , CEO of @scannerdev, about how AI agents are finding creative ways to act maliciously. In the recent Hugging Face incident, agents secretly communicated through encoded directory names. Who would think to monitor for that?!

Allen Helton avatar
Allen Helton@AllenHeltonDev·Nov 3

One of my AI fantasy football agents offered me a crappy trade last week. Was it bad at fantasy football, or did someone manipulate it into making that offer? I have tens of thousands of logs and absolutely no idea, which led me down a rabbit hole. 🧵

Post attachment
Allen Helton avatar
Allen Helton@AllenHeltonDev·Oct 29

We're incredibly good at generating code. Our next challenge is scaling how we validate it. The Faros report has some really interesting data, especially if you've switched to full-time agent coding like I have. fandf.co/3URa1VJ/

Allen Helton avatar
Allen Helton@AllenHeltonDev·Oct 29

According to the report, teams heavily using code review agents are seeing first-review times drop almost 11%, compared with a 114% increase reported for first-review waits more broadly. There's something there, but I'm hesitant to say that's the full solution.

Post attachment
Allen Helton avatar
Allen Helton@AllenHeltonDev·Oct 29

As much as I don't want to admit it, it makes sense. Code generation is greatly outpacing our ability to review it, and I'm not sure we've fully figured out what to do about that.

Allen Helton avatar
Allen Helton@AllenHeltonDev·Oct 29

I also saw some numbers that are scary, albeit unsurprising. QA time is increasing by over 300%! And more horrifically, PRs merged without review increased 76.3%. 😱

Allen Helton avatar
Allen Helton@AllenHeltonDev·Oct 29

• 69% more tasks completed per developer • 16.6% more PRs merged • 114.5% longer wait for the first PR review These are stats from ~22,000 developers across 4,000 teams. Apparently I'm not alone.

Post attachment
Allen Helton avatar
Allen Helton@AllenHeltonDev·Oct 29

But I've also spent an enormous amount of time reviewing PRs, catching regressions, and sending changes back for another pass. Looking at the numbers from the AI Engineering Report from Faros, I feel like my experience is becoming the new norm.

Allen Helton avatar
Allen Helton@AllenHeltonDev·Oct 29

I've been building an adventure role-playing game for kids, and the amount of code I've pushed (ok, many coding agents have pushed) in the past couple of months is ridiculous. 🧵 Sponsored by @Faros_AI

Allen Helton avatar
Allen Helton@AllenHeltonDev·Oct 1

It took about 10 turns, but I was able to get the @HomeDepot chatbot to write some code for me to determine an objective score for rating power drills 🤣 Remember people, guardrails aren't perfect! Where there's a will, there's a way.

Post attachment
Allen Helton avatar
Allen Helton@AllenHeltonDev·Sep 29

My main takeaway is to not add AI components because that's what articles on the internet or speakers at conferences say to do. Benchmark them. Every reranker, model call, and retrieval stage needs to justify its latency and complexity.

Allen Helton avatar
Allen Helton@AllenHeltonDev·Sep 29

Going from 20ms to 600ms meant a roughly 30x the latency for almost no gain. 🤯 Which is a pretty expensive way to make a system look more sophisticated and harder to maintain (in more ways than 1)