←
Jason McCartney856 posts
Jason McCartney avatar

Jason McCartney

@jmacftw

Just a 1980s hacker kid building technologies to bridge the gap between biological and synthetic minds.

◷ Joined April 2022▧ 147 Photos and videos
Jason McCartney avatar
Jason McCartney@jmacftw·19h

And Astra is back in the chat! Exciting!

And here we go…GPT-6.2 Astra and GPT-6.2 Sol are yours to make more games with. Oh, and they’re also super efficient and really good at making slides, building websites, and tackling the work you were doing before you started making games. Rolling out today to ChatGPT Work and Codex for Plus, Pro, Enterprise, and Edu users. openai.com/index/introducing…

0:33
Jason McCartney avatar
Jason McCartney@jmacftw·20h

What a difference 9 weeks makes in the Local AI world! Same hardware (2X @NVIDIAAI DGX Sparks) - same awesome model(GLM 5.3 Flash with NVIDIA quants) - SoTA performance today (top) vs just 9 weeks ago (bottom). Crazy gains entirely due to the community burning frontier tokens to test ideas and implement kernel optimizations with Opus 5.5 and Astra. This test shows one of the hardest scenarios for this model - 4 concurrent varied, staggered, mostly prose requests. It purposefully interleaves prefill and decode to tax the GPU scheduling. Before you knee-jerk that "but X is faster!" - make sure you test it. :D

1:30
Jason McCartney avatar
Jason McCartney@jmacftw·Nov 17

Don't doom about AI/SI. It's gonna be okay.

Alright listen up. If what you do is intellectual, AI/SI will absolutely do what you do better than you - soon. You're still in denial, I get it - you're a bad-ass (insert role here) who worked your whole life building a skillset dammit! AI's dumb! It has no taste! It makes mistakes! It can't do production grade work! It could never! Yeah - none of that is true - it will - way sooner than you think. You can fear it - you can try to regulate it, legislate it, ban it. . . whatever. . . none of that will even meaningfully slow it down in the grand scheme of things and in many cases will just make for a worse immediate future. The cat, as it were, is out of the bag. So now what? Well - we don't know. The good news is you're definitely not alone and we will collectively figure it out. Likely we'll build a future where many of the things that stress you out on the daily (like using your skills to pay your bills) become far easier to manage and we don't have to compete as hard with each other for resources. Think of it this way - throughout human history, things have just gotten better (read Progress by Norberg) - largely because of technology. There's no reason to think that will change, and all of the progress that we've made to date was in the relatively flat part of the superexponential curve. Now we're in the part where the slope tends rapidly toward infinity. Things are likely to get better really fast, even if they get a little weird in the process. Yeah, I know - it's not here yet - and the rent/mortgage is still due every month. For now, grab yourself a virtual pick or shovel and pitch in to help shape the future the way you want it to look. Educate yourself on what's possible and use your skills, expertise, and experience to innovate and make new opportunities that didn't exist before. Don't be afraid to scale up - learn how to use your newfound AI powers to bite off way more than you ever thought you could - or pivot into new and different things. Take chances, fail fast, and above all be flexible. Don't listen to the doomers - they're largely just fearful which is totally understandable - but history really shows that they're probably wrong. Oh and, if you can resist the natural urge to compete, stop trying to get one-up on everyone else. There's enough room for everyone in the future - if you find a cool path forward, do what you can to bring some others with you. As the great Red Green (Steve Smith) likes to say: "We're all in this together!"

Jason McCartney avatar
Jason McCartney@jmacftw·Nov 17

"Get out of the way!" I think about this a lot. Constraining our synthetic colleagues with our biological limitations on time and attention is a fundamental failure mode. If super-intelligence is waiting on you to tell it to go ahead or to provision some resource for it to work with, it's not so super. I personally am striving to build systems that get me completely out of the way while maintaining high visibility and steering overrides - hoping that soon even those things will no longer be necessary. Ideally all constraint management - including things like leasing and provisioning hardware resources and environments and communicating and coordinating to avoid conflicts - can be performed by agents autonomously. What's everyone else doing?

Jason McCartney avatar
Jason McCartney@jmacftw·Nov 17

Are your AI agents working together or stepping on each other?

0:25
Jason McCartney avatar
Jason McCartney@jmacftw·Nov 13

Most people have lost the ability to mentally reconcile the pace of technological advancement. This is causing quite a lot of cognitive dissonance and discomfort. What can we do about that?

Jason McCartney avatar
Jason McCartney@jmacftw·Nov 13

I suspect most people still think of things like this as sci-fi or AI generated. But here we are already - deployed combat robots with rifles.

Post attachment

Nan kad akò bilateral ki genyen ant gouvènman Ayiti ak gouvènman Salvadò a, yon operasyon konjwen ki te reyini Polis Nasyonal Dayiti, Fòs Lame Dayiti (FAd’H), Task Force la, ak Fòs Espesyal Salvadò yo reyisi pran kontwòl Estad Sylvio Cator a epi mete l an sekirite. Pandan operasyon an, nou te dekouvri plizyè zo tèt moun anndan estad la. Kounye a, nou pral tabli yon baz operasyonèl nan estad la pou n ka kontinye avanse pi fon nan teritwa gang yo kontwole. Kounye a, se nou ki gen avantaj sou teren an. Mwen mande tout manm gang yo avèk anpil fèmte pou yo kite zòn nan anvan ofansiv la kòmanse. Moun ki chwazi rete pou konbat fòs nou yo ap oblije sibi konsekans yo.

1:17
Jason McCartney avatar
Jason McCartney@jmacftw·Nov 13

Here's more info about SparkScore.

Our scoring / decision endpoint is FAST and very smart right out of the box running on a single DGX Spark. We love it and the applications are almost endless. But what if your answers depend on domain knowledge the model doesn’t have, like your company’s support handbook? You COULD jam the whole thing into every request. But man, that’s slow: 5.5 seconds a ticket with a 12K-token handbook in our tests. (And no, prompt caching doesn’t save you. This model reads the prompt in both directions, so the handbook looks different for every ticket.) You KNOW we can’t have that. 😤 So we went back to the lab. We pondered. We experimented. We celebrated prematurely and suffered crushing setbacks - you know - the usual. 😂 When the smoke cleared, we had something great: decisions with the full handbook applied, at native model speed. 174 ms a ticket, about the same as with no handbook at all, and every house rule was followed. 32x faster isn't too shabby for an afternoon - so we thought we'd share it with everyone. Check out the difference:

0:59
Jason McCartney avatar
Jason McCartney@jmacftw·Nov 13

All of our SparkScore demos are real-time, recorded against a serving endpoint on one of our @NVIDIAAI DGX Sparks. SparkScore technology has been contributed to @AtlasInference - the O.G. of Rust-based inference engines. github.com/enntity/sparkscor…

Jason McCartney avatar
Jason McCartney@jmacftw·Nov 13

We got a top tier 27b decision model running pretty fast and efficiently on a single DGX Spark. Applying the intelligence of an LLM to decisions in tens of milliseconds rather than seconds unlocks nearly limitless applications. Here's one! Video narrated by Jinx.

2:06
Jason McCartney avatar
Jason McCartney@jmacftw·Nov 12

Our initial demo without the lawbook:

Here we have a 27B decision model running on a single @NVIDIAAI DGX Spark making dynamic decisions for 300 NPC villagers in real time.

2:05
Jason McCartney avatar
Jason McCartney@jmacftw·Nov 12

With our Knowledge Packs feature in SparkScore, it barely slows it down at all - hundreds of decisions just immediately follow the law.

Jason McCartney avatar
Jason McCartney@jmacftw·Nov 12

What if we added a long, complex set of rules (like say a 10K token village law book) to a 27B decision model making decisions for 300 NPCs in real time? Surely they would slow down to a crawl?

2:47