←
@cdleary105 posts
@cdleary avatar

@cdleary

@cdleary

computers/compilers/accelerators, mostly • Intelligence Processors at OpenAI 🌶️

◷ Joined November 2008▧ 4 Photos and videos
@cdleary avatar
@cdleary@cdleary·Sep 14

more on GPT/codex usage in Jalapeño and some of “where we’re going from there”… as article says the models really grok’d XLS, but it’s made huge strides in systemverilog as lingua franca, if you last tried before Astra it’s time for another look…

How OpenAI Used Its Own LLMs to Design Its Jalapeño Chip spectrum.ieee.org/llms-for-c…

@cdleary avatar
@cdleary@cdleary·Sep 4

egg, and panko, on our face we know you have choices when it comes to spicy codenames and we're committed to doing better

There is a lot of deserved hype for how great OpenAI's Jalapeno ASIC and system will be. However, there is one major flaw that no one is talking about at all... (1/6)🧵

Post attachment
@cdleary avatar
@cdleary@cdleary·Aug 30

for folks who didn't see the slide on MLA we presented at @hotchipsorg the whole idea is you can speedrun functional description to obtainable roofline all within the semantics of the machine / primitives exposed in the gluon kernel language

Post attachment
@cdleary avatar
@cdleary@cdleary·Aug 30

if you sat down and really studied it you could put it all back together, and the semantics are naturally limited by the kernel language, but part of the idea of ai-qua-kernel-generator is that you don't want to have to do that

Muzaffer Kal @🏡 🦆⏩ avatar
⟳ @cdleary retweeted
Muzaffer Kal @🏡 🦆⏩@MuzafferKal_·Aug 30

How is this any different than looking at the output of a compiler and saying “I don’t know what that does line by line?” Most people can’t even read assembly code.

SemiAnalysis' Jordan Nanos reveals the OpenAI engineers scrolling their own kernel code had no idea what it did line by line, and it didn't matter because the AI does "It's not slop, because it performs, but it's literally just AI-generated assembly basically puked out in Gluon, this low-level kernel programming language that they built on top of Triton, which uses this really interesting programming model." "The point is, the guys who were scrolling through this code with us, it was kind of clear that they know a whole bunch about hardware, they know a whole bunch about the concepts in the system, and they just have no idea what this MLA kernel that they're showing us for DeepSeek actually does." "You can go line by line and it's like nope, nope, nope." "But it doesn't matter. The AI understands it, the AI tests it, and you see the results. It produces correct kernels that perform really well." "And I think this is just a sign of what's to come again." "That actual code is not necessarily something that a human has to reason about deeply, if the AI knows how to manipulate the data movement and the processing elements on the hardware that you've given it." ________ More takeaways from OpenAI's Jalapeno development: firesidealpha.substack.com/8…

1:12
@cdleary avatar
@cdleary@cdleary·Aug 30

it is quite analogous to not knowing exactly what assembly comes out of the compiler at -O3 and why, even if you wrote the original "functional spec" of C++ code that fed the compiler

@cdleary avatar
@cdleary@cdleary·Aug 30

don't forget to be excellent to one another, please...

@cdleary avatar
@cdleary@cdleary·Aug 30

just because i'm seeing ad hominem attacks for some godforsaken reason i was the one who was scrolling and @JordanNanos is 100% right ai is what optimized it into its roofline achieving form

SemiAnalysis' Jordan Nanos reveals the OpenAI engineers scrolling their own kernel code had no idea what it did line by line, and it didn't matter because the AI does "It's not slop, because it performs, but it's literally just AI-generated assembly basically puked out in Gluon, this low-level kernel programming language that they built on top of Triton, which uses this really interesting programming model." "The point is, the guys who were scrolling through this code with us, it was kind of clear that they know a whole bunch about hardware, they know a whole bunch about the concepts in the system, and they just have no idea what this MLA kernel that they're showing us for DeepSeek actually does." "You can go line by line and it's like nope, nope, nope." "But it doesn't matter. The AI understands it, the AI tests it, and you see the results. It produces correct kernels that perform really well." "And I think this is just a sign of what's to come again." "That actual code is not necessarily something that a human has to reason about deeply, if the AI knows how to manipulate the data movement and the processing elements on the hardware that you've given it." ________ More takeaways from OpenAI's Jalapeno development: firesidealpha.substack.com/8…

1:12
@cdleary avatar
@cdleary@cdleary·Aug 27

beautifully done walkthrough of the @hotchipsorg talk slides by @IanCutress /the/ place to watch until the hotchips talks are on youtube note the Gluon kernel language is the highest-performance low-level-primitives-exposed layer in the Triton ecosystem triton-lang.org/main/gluon/i…

I love the taste of jalapeños. Here's my take on the @OpenAI presentation chip that claims to be faster than @nvidia. youtu.be/Ic0kYWjffjI

Post attachment
Greg Brockman avatar
⟳ @cdleary retweeted
Greg Brockman@gdb·Aug 26

ai for chip design is underrated

OpenAI decided to keep the chip team small and use AI for a an improvement loop Early stages of Hardware in the loop RSI?

Post attachmentPost attachment
Sam Altman avatar
⟳ @cdleary retweeted
Sam Altman@sama·Aug 25

we made a chip and it is fast

OpenAI avatar
⟳ @cdleary retweeted
OpenAI@OpenAI·Aug 25

We plan to begin deploying Jalapeño in OpenAI’s compute infrastructure by year-end. It’s the first step in a multigenerational roadmap: Gen 2 is deep in development, and Gen 3 is taking shape. Each generation will push efficiency and speed further. openai.com/index/jalapeno-fi…

Greg Brockman avatar
⟳ @cdleary retweeted
Greg Brockman@gdb·Aug 25

inference numbers published for jalapeno, team did an amazing job openai.com/index/jalapeno-fi…

OpenAI: "Today, we shared the first measured performance results from Jalapeño, OpenAI’s first custom inference chip. On InferenceX, a public benchmark using GPT‑OSS 120B, Jalapeño delivered more peak throughput per kilowatt and lower token latency than the commercial systems in the comparison. It also performed strongly on DeepSeek R1 and Kimi K2, showing that its gains extend across model families." "This is Jevons paradox: greater efficiency makes more uses worthwhile, expanding consumption and creating new economic activity through more work completed, better decisions, more products launched, and more revenue generated." "AI played a direct role in Jalapeño’s development, enabling the team to move from initial design to tapeout in nine months by exploring implementations, shortening design, measurement, and verification loops, and continuously iterating on model workloads. AI also helped optimize the chip’s arithmetic circuits, allowing the team to fit more compute performance into the chip on schedule." "We plan to begin deploying Jalapeño within OpenAI’s compute infrastructure by the end of the year. It is the first generation of a multigenerational roadmap: Gen 2 is deep in development, and Gen 3 is taking shape. Each generation will build on what we learn and further advance both efficiency and speed." "As we prepare for deployment, we are continuing production qualification, maturing the software, preparing to operate Jalapeño at scale, and validating performance across more models."

Post attachmentPost attachmentPost attachment