mmissoldtw ←
brian stevens752 posts
brian stevens avatar

brian stevens

@addvin

CEO, Neural Magic. Ex VP, CTO of Google Cloud and EVP, CTO of Red Hat, RPI and UNH alumn, marathoner, ironman, ADK MT 46er.

◷ Joined January 2009▧ 59 Photos and videos
brian stevens avatar
brian stevens@addvin·22 May 2025

Introducing llm-d on stage at Red Hat Summit was truly a privilege ...

Red Hat AI@RedHat_AI20 May 2025

LLM inference is too slow, too expensive, and too hard to scale. 🚨 Introducing llm-d, a Kubernetes-native distributed inference framework, to change that—using vLLM (@vllm_project), smart scheduling, and disaggregated compute. Here’s how it works—and how you can use it today:

Red Hat AI avatar
⟳ brian stevens retweeted
Red Hat AI@RedHat_AI·Aug 10

Muse-Glimmer 30B, now in FP8-block. Our quantized checkpoint of the multimodal model, at roughly half the memory and disk. Block-wise FP8 on the linear layers, vision tower kept in full precision. Quantized with LLM Compressor, serve on @vllm_project. Evals coming shortly. huggingface.co/RedHatAI/Muse…

Today we're also opening the weights for Muse Glimmer, a great 30B parameter dense model that can run locally. Soon we'll also release the weights for Muse Spark 1.2, our latest foundation model. Meta is a strong supporter of open source and I'm proud of these releases. Congrats to @alexandr_wang and the MSL team for all your great work on these models.

Red Hat AI avatar
⟳ brian stevens retweeted
Red Hat AI@RedHat_AI·Jul 28

Kimi K3 open weights landed yesterday. By that afternoon, Red Hat AI Inference was serving it on one 8x B300 node. 2.8T params. Day-0 preview images exist so you can experiment the moment weights drop. 👇

Red Hat AI avatar
⟳ brian stevens retweeted
Red Hat AI@RedHat_AI·May 18

🇬🇧 London, June 10. @vllm_project & @_llm_d_ Inference Meetup, hosted by Red Hat AI, @nvidia, and @SteliaAI. Talks on vLLM updates, speculative decoding, llm-d in production, AI safety, and more. Plus food, drinks, and the people building this stuff. luma.com/iuecyow4

Red Hat AI avatar
⟳ brian stevens retweeted
Red Hat AI@RedHat_AI·Apr 25

Llama 70B as a cloud endpoint costs exponentially more than Llama 8B. For teams where a smaller model meets the quality bar, that gap is hard to ignore. And with INT4 quantization: 4x smaller, 2x faster, less than 1% accuracy loss. The right model isn't always the biggest one. redhat.com/en/blog/when-less…

Red Hat AI avatar
⟳ brian stevens retweeted
Red Hat AI@RedHat_AI·24 Dec 2025

Calling Boston area startups building with AI. 🤙 We're kicking off 2026 with the first event in a new monthly, in person hackathon series hosted by @RedHat and @IBM in Boston’s Seaport District. This one day hackathon is designed specifically for local startups that want to move faster from idea to working prototype. Instead of a fixed theme, you bring a real AI problem your team is actively facing. We help you build a proof of concept using open source, enterprise ready templates from aitemplates.io, including MCP Server, AI Agent, and UI templates. What you will get: ⚡ Rapid prototyping without boilerplate 🧠 Hands on guidance from Red Hat AI architects 🤝 Connections with other Boston based AI startups and ecosystem partners If you are a Boston startup looking to turn an AI challenge into something real, this is for you. Event details are shared after registration. Register now: luma.com/i3q8df0x

SemiAnalysis avatar
⟳ brian stevens retweeted
SemiAnalysis@SemiAnalysis_·24 Oct 2025

The @RedHat_AI team contributes a lot to vLLM and does amazing work for the open-source community. Great to see vLLM performing so well compared to TRT-LLM on H200! vLLM comes pretty close to B200, with the @NVIDIAAI team working on closing the gap for GPTOSS within the next couple of updates.

Post attachment
Red Hat AI@RedHat_AI24 Oct 2025

InferenceMAX, vLLM TPU, compressed-tensors, MoE support via transformers, DeepSeek-OCR, and more. Here’s what’s new in the @vllm_project community over the past two weeks:

12:26
Red Hat AI avatar
⟳ brian stevens retweeted
Red Hat AI@RedHat_AI·24 Oct 2025

InferenceMAX, vLLM TPU, compressed-tensors, MoE support via transformers, DeepSeek-OCR, and more. Here’s what’s new in the @vllm_project community over the past two weeks:

12:26
Red Hat AI avatar
⟳ brian stevens retweeted
Red Hat AI@RedHat_AI·22 Sep 2025

4 tracks. 12 sessions. 1 day of learning. Join us on Oct. 16 for Red Hat AI Day of Learning, a free virtual event for developers, engineers & practitioners. Tracks: ⚡ Fast & efficient inference 🎯 Model customization 🤖 Agentic AI 🌐 Scaling AI over hybrid cloud Sessions include: · Intro to vLLM and how to get started · Model optimization with LLM Compressor · Lossless LLM inference acceleration w/ Speculators · End-to-end model customization · Synthetic data generation and data processing · Continual learning of LLMs with Training Hub · Build open source agentic AI solutions · Intro to Model Context Protocol (MCP) · Intro to Llama Stack · Intro to distributed inference · Distributed inference with llm-d · Scaling AI Infrastructure 👉 Register free: redhat.com/en/events/webinar…

Red Hat AI avatar
⟳ brian stevens retweeted
Red Hat AI@RedHat_AI·29 May 2025

Thanks to the @lmcache team for joining forces with Red Hat on llm-d! llm-d is a new open source project for scalable, efficient distributed LLM inference with @vllm_project. Learn more about our collaboration here: blog.lmcache.ai/2025-05-22-r…

Mark Collier 柯理怀 avatar
⟳ brian stevens retweeted
Mark Collier 柯理怀@sparkycollier·21 May 2025

Really excited to see the emergence of llm-d @addvin ! Inference is the biggest workload in human history and the open source tools need to keep evolving to serve it

The llm-d project is a major step forward for the #opensource AI ecosystem, and we are proud to be one of the founding contributors, reflecting our commitment to collaboration as a catalyst for innovation in generative AI. As generative and agentic AI continue to evolve, scalable, high-performance inference will be critical to unlocking their full potential. That’s why we’re partnering with @RedHat and other contributors to grow the llm-d community and accelerate its capabilities—powered by our contributions, including innovations from NVIDIA Dynamo such as NIXL. 🔗 Explore and contribute on GitHub: nvda.ws/3FlttSL 📰 Read the launch blog: nvda.ws/3FgF0Tn 🎙️ Hear from NVIDIA’s VP of Engineering & AI Frameworks, Ujval Kapasi → nvda.ws/45pyVhU

Post attachment
NVIDIA AI Developer avatar
⟳ brian stevens retweeted
NVIDIA AI Developer@NVIDIAAIDev·20 May 2025

The llm-d project is a major step forward for the #opensource AI ecosystem, and we are proud to be one of the founding contributors, reflecting our commitment to collaboration as a catalyst for innovation in generative AI. As generative and agentic AI continue to evolve, scalable, high-performance inference will be critical to unlocking their full potential. That’s why we’re partnering with @RedHat and other contributors to grow the llm-d community and accelerate its capabilities—powered by our contributions, including innovations from NVIDIA Dynamo such as NIXL. 🔗 Explore and contribute on GitHub: nvda.ws/3FlttSL 📰 Read the launch blog: nvda.ws/3FgF0Tn 🎙️ Hear from NVIDIA’s VP of Engineering & AI Frameworks, Ujval Kapasi → nvda.ws/45pyVhU

Post attachment
Red Hat AI avatar
⟳ brian stevens retweeted
Red Hat AI@RedHat_AI·26 Feb 2025

DeepSeek’s Open Source Week drops A LOT of exciting goodies! We’re hosting vLLM Office Hours tomorrow—learn what they are, how they integrate with vLLM, & ask questions! Date: Thursday, Thu, Feb 27 Time: 2PM ET / 11AM PT Register: neuralmagic.com/community-of… #deepseek #ai

Post attachment
Matt Hicks avatar
⟳ brian stevens retweeted
Matt Hicks@matthicksj·13 Jan 2025

At @RedHat, we believe the future of AI is open. That's why I'm incredibly excited about our acquisition of @NeuralMagic. Together, we're furthering our commitment to our customers and the open source community to deliver on the future of AI—and that starts today.

Red Hat@RedHat13 Jan 2025

Today, Red Hat completed the acquisition of @NeuralMagic, a pioneer in software and algorithms that accelerate #genai inference workloads. Read how we are accelerating our vision for #ai’s future: red.ht/408kJ8K.

0:54