←
lbr33 posts
lbr avatar

lbr

@teleogen

Enterprise AI - AI agents harness - Local LLMs

◷ Joined June 2009
lbr avatar
lbr@teleogen·Jun 26

The most bullish thing for the AI sector is to actually cause measurable deflation.

lbr avatar
lbr@teleogen·May 15

Generative AI is great at writing code. It’s terrible at deleting it. The next frontier is Subtractive AI: systems rewarded for what they remove and simplify — not what they add.

lbr avatar
lbr@teleogen·Apr 20

8/8 TL;DR: Looped transformers = better quality per token, but 3-5× more compute. Smaller models. Higher latency. More GPUs needed for same throughput. The frontier is getting deeper, not wider. And the hardware bill is coming.

lbr avatar
lbr@teleogen·Apr 20

7/8 Investment takeaway: Don't just watch VRAM demand. Watch: • Compute density per token • Cache efficiency • Interconnect tech • Specialized recurrent hardware The next hardware cycle won't look like the last one.

lbr avatar
lbr@teleogen·Apr 20

6/8 Here's the kicker: Memory bandwidth pressure goes DOWN (weight reuse is the key benefit). But compute demand goes UP 40-60%. This flips the script on what we thought would bottleneck AI scaling.

lbr avatar
lbr@teleogen·Apr 20

5/8 The hardware winners: • HBM4 memory (state management) • Custom AI chips (recurrent-optimized) • L2/L3 cache (weight reuse efficiency) • Better interconnects (NVLink, etc.) The losers: • Low-end GPUs • CPU inference • Traditional cloud instances

lbr avatar
lbr@teleogen·Apr 20

4/8 This is a throughput-for-quality swap. Datacenters: need MORE GPUs to maintain same token/sec. Edge devices: win (smaller models fit on consumer hardware). Latency-sensitive apps (chat, real-time) suffer. Batch processing (research, analysis) benefits.

lbr avatar
lbr@teleogen·Apr 20

3/8 The tradeoff is brutal: ✅ Less VRAM needed (fewer unique weights) ✅ Less memory bandwidth (weights reused) ✅ Smaller storage footprint ❌ 3-5× more FLOPs per token ❌ 3-5× higher latency (loops are serial) ❌ Harder to scale across GPUs

lbr avatar
lbr@teleogen·Apr 20

2/8 Traditional transformers: 100+ unique layers, loaded once per token. Looped transformers: 10-20 layers × 5-10 loops = same "effective depth" with fewer unique weights. Result? 30-50% smaller model files. But you pay for it elsewhere.

lbr avatar
lbr@teleogen·Apr 20

1/8 Looped transformers are coming. OpenMythos, recurrent-depth architectures, weight recycling — the frontier labs are all exploring this. But here's what nobody's talking about: the hardware implications are massive. Let me break it down. 🧵

Introducing OpenMythos An open-source, first-principles theoretical reconstruction of Claude Mythos, implemented in PyTorch. The architecture instantiates a looped transformer with a Mixture-of-Experts (MoE) routing mechanism, enabling iterative depth via weight sharing and conditional computation across experts. My implementation explores the hypothesis that recursive application of a fixed parameterized block, coupled with sparse expert activation, can yield improved efficiency–performance tradeoffs and emergent multi-step reasoning. Learn more ⬇️🧵

Post attachment
lbr avatar
lbr@teleogen·Mar 13

This is how vLLM got started. The best explanation by far!

Avi Chawla @_avichawla Mar 11 Article Paged Attention in LLMs, clearly explained! When you’re serving LLMs at scale, memory becomes your bottleneck before compute does. The problem isn’t a lack of GPU memory; it’s how inefficiently we use it. Let’s understand why this happens and

lbr avatar
lbr@teleogen·Mar 13

Such a big deal. Alternatives are @UnslothAI and @axolotl_ai

Introducing Trellis for Kimi K2 Thinking. It's post-training code that's 50x faster than the best single-node open-source version and 2x cheaper than training APIs. After safety testing, we're open-sourcing it, giving builders the best tools to customize a frontier model. 🧵

Post attachment
lbr avatar
lbr@teleogen·Mar 1

Curious to see whether they will release medium-size models to compete with Alibaba Qwen3.5 series

Finally, V4 is coming * Set to release next week ahead of the Two Sessions * Optimised for Chinese chips led by Huawei and Cambricon * Multi-modal and early tests results are “impressive” Exclusive: DeepSeek to release long-awaited AI model as.ft.com/r/cd6577d9-0633-49…