←
SGLang803 posts
SGLang avatar✓

SGLang ✓

@sgl_project

Run LLMs fast at any scale 🔗 github.com/sgl-project/sglan… Join our community slack.sglang.io For AI tech blogs & deep-dives 👉 @lmsysorg

◷ Joined May 2025▧ 143 Photos and videos
SGLang avatar
SGLang@sgl_project·19h

2,000+ people. Two days. The first SGLang Summit. Together, we announced SGLang v1.0, released Miles v0.2, and launched the SGLang Consortium with nine founding members: @nvidia, @AMD, @intel, @Google, @Meta, @thinkymachines, @Cloudflare, @modal, and @radixark. SGLang is now used by 332 organizations. At Meta, it powers production inference for products serving billions of users. Teams from Modal, Cloudflare, LinkedIn, and others shared their production lessons. Across 90+ technical talks and six hands-on workshops, the community explored what comes next for inference, reinforcement learning, and AI agents. To everyone building, deploying, and contributing: thank you for making this possible. Explore the announcements, engineering lessons, and moments from SGLang Summit 2026 in the link below.

Post attachment
SGLang avatar
SGLang@sgl_project·7h

🎉 SGLang has Day-0 support for Hy4 flash from @TencentHunyuan, a 224B multimodal model with 1M context and image input. - 1/13 the per-task cost of Hy4 preview, KV cache 1/6 the size of Hy4 - DFly2 speculative decoding with zero extra KV memory - LatentMoE + Engram: 16 of 512 experts per token, plus bigram and trigram embeddings Try it now with SGLang!

Post attachmentPost attachment
SGLang avatar
SGLang@sgl_project·8h

SGLang now supports Sage S2.1 Flash, a multimodal model for scientific research! @sageai_official > Multi-round research with retrieval, coding, and tool use. 10+ scientific input types: molecules, crystals, proteins > #1 on MLE-bench Lite (90.3), OfficeQA Pro, Harvey LAB, SpreadsheetBench 2 Run it now with SGLang!

Post attachmentPost attachmentPost attachment
SGLang avatar
SGLang@sgl_project·Nov 17

~2,500 tokens/sec at batch size 1 for DeepSeek v4.1 Flash on Vera Rubin, up to 1.7× faster than GB300 in Baseten's tests 👀 Great to see these results shared at SGLang Summit last week. We're working with the Baseten team to bring these low-latency optimizations to open source!

0:16
SGLang avatar
SGLang@sgl_project·Nov 14

RL rollouts don't have to run on the same hardware as training. @Weyaxi and the @aiand_ team showed how Miles can train on B300s, while SGLang serves rollouts across NVIDIA B300/H200 and AMD MI355X/MI300X. Great work, and thanks for sharing at SGLang Summit!

0:36
SGLang avatar
SGLang@sgl_project·Nov 14

That's a wrap on SGLang Summit 2026! Incredible energy all around: the main stage, the expo hall, the booths, the workshops, and all the conversations that happen around them! To everyone in the community: THANK YOU for the wonderful memories! You make SGLang better 🧡🧡🧡

Post attachment
GeekPark avatar
⟳ SGLang retweeted
GeekPark@GeekParkHQ·Nov 14

Perplexity (@perplexity_ai) CEO Aravind Srinivas (@AravSrinivas) spent 15 minutes at SGLang Summit (@sgl_project) talking about “Building the Next Computer.” Some notes from the talk: 00:44 - How it started. Perplexity built a Slack agent in January where employees talked to a bot more than each other. Marc Andreessen texted Aravind about a company where employees mostly talked to a bot instead of each other. “That was basically us.” 01:35 - Why Computer? Almost SuperAgent or MasterAgent, but he chose Computer, a word that once referred to people doing math for astronomers. 04:20 - Search as code. Fetch, chunk, embed, retrieve, all exposed through a Python SDK so agents can build their own search pipelines. 08:15 - SPACE. Perplexity now runs millions of concurrent sandboxes every day. The platform is open source, including fixes from red-teaming. 12:22 - Local + cloud. A fine-tuned Qwen 27B scored 59.6% locally on 89 tasks; calling Opus 5 only when needed raised it to 73%. 13:20 - Lily. Built by an intern who joined full-time, this Apple Silicon inference engine beats MLX-LM by ~22% on prefill and ~30% on decode on an M5 Max. He ended by telling everyone they're still hiring.

15:41
SGLang avatar
SGLang@sgl_project·Nov 13

💡 Conversations shaping the next frontier are happening at SGLang Summit Catch snippets from @LipBuTan1's fireside chat with @ying11231 👇

0:56