←
Ani Rahul832 posts
Ani Rahul avatar

Ani Rahul

@Ani_da_dev

high data circle

◷ Joined November 2023▧ 202 Photos and videos
Ani Rahul avatar
Ani Rahul@Ani_da_dev·20h

BREAKING: NVIDIA $NVDA's board does not intend to reinstate Jensen Huang as CEO, per their CHAIRMAN HARVEY JONES on the earnings call right now Big drop.

Post attachment
dhruv bhatia avatar
⟳ Ani Rahul retweeted
dhruv bhatia@dhruvbhatia0·Nov 16

you dont work on bleeding edge stuff if your slack isnt littered with this guy

Post attachment
Ani Rahul avatar
Ani Rahul@Ani_da_dev·Nov 12

The dark forces are coming after my portfolio

ask better questions. look at the tenacity of the headlines from two news publications recently against Jensen after the election. this is a dedicated planned coup against him by unknown forces. he needs to fight. they are stacking the deck to emotionally overwhelm him into resigning.

Ani Rahul avatar
Ani Rahul@Ani_da_dev·Nov 12

Actually very bullish on AWS after seeing this

We just shipped AWS Emulator, which gives coding agents a local copy of AWS to test against. Early in my career, we built test environments like this by hand, one system at a time. Now, agents are writing code far faster than any team can check it in production. Matt Goldberg, a Senior Principal Engineer on our Developer Tools team, wanted to know whether agents could build and test those environments themselves. He gave @kirodotdev a single prompt, and about 15 minutes later it came back with a first version of an SQS emulator. Turing that into a robust solution took much more work Here’s what I think this means for engineers today: allthingsdistributed.com/202…

Ani Rahul avatar
Ani Rahul@Ani_da_dev·Nov 11

Lmao one Opus 5.5 prompt to figure out that its an OpenCode fork, and they got SOTA on terminal bench by hard coding in hints. No end to the larp

Post attachment

We achieved SOTA on Terminal Bench 2.1 using a DeepSeek v4.1 Flash model. 92.2% at $0.08 cost per task. Previously Opus 5.5 was the leader at 87.6% at $0.50 cost per task (vals.ai/benchmarks/terminal-…). Now the obvious question is how did we do it? How can you possibly improve a Flash model to beat frontier models like Opus, Fable, and Astra? The truth is, we over-obsess about models when they are only half of the equation. The other half is the code that wraps around the model. In fact all of the most revolutionary products (Instinct, Muse, Claude Code, OpenClaw, etc.) are all just harnesses optimizations. Rather than try to build a new model or fine-tune, we built a harness that would make: Flash Models —> Frontier Results Frontier Models —> Beyond Frontier Results Did I mention it’s also ~6x faster than Claude Code? And we’re now live. Go Check it out! Trymidas.io

Post attachment
Proximal avatar
⟳ Ani Rahul retweeted
Proximal@ProximalHQ·Nov 6

Congratulations to @thinkymachines! We've been able to help evaluate Inkling 2 Small's autoresearch capabilities - it is a strong model for long-horizon research tasks!

Congratulations to @thinkymachines! I love to see the level of thoroughness and attention to detail when it comes to Inkling's evaluation methodology. Also excited to see some of the auto-research evals we have been working on featured in the blog!

Post attachment
Ani Rahul avatar
Ani Rahul@Ani_da_dev·Nov 6

Cool work!

Today, in the first of a series of blog posts we’re calling “Behind the Grove”, we’re offering a look into some of the technical work we do at DeepGrove. Our first post is about our pretraining infrastructure, Grove. Grove is built around 2 backends: Grove-FSDP and DDP. We detail how we optimized both backends, culminating in an over 2x speedup over public baselines. A production-scale pretraining run of Maple 20B-A1B (deepgrove.ai/maple-preview) on 4T tokens takes a mere 2 weeks on 64 H100s. We view infrastructure as the backbone of our work, and as one of the many pieces that help us build toward the frontier of efficiency and performance. To read the full post, visit deepgrove.ai/infra.

Post attachment
Ani Rahul avatar
Ani Rahul@Ani_da_dev·Nov 2

Fair take but margin can't be more than 100%

Post attachment

The reason it’s ONLY 85% margin is because of open source. There should be no doubt in your mind if there was not open models capable of doing much of the same tasks being served at only 25-40% margin Anthropic and other closed labs would be happy to take 200%+ margin. It’s competitive pressure and it’s good for everyone.

def1ant avatar
⟳ Ani Rahul retweeted
def1ant@0xdef1ant·Oct 29

just FYI right now you can buy a Dell Pro Max T2 Desktop with an RTX 6000 Pro Blackwell card for less than the price of the card itself

Post attachmentPost attachment
Ani Rahul avatar
Ani Rahul@Ani_da_dev·Oct 27

I think the people who want fast tokens usually also want the best models? I don’t really see the market for Kimi K3 ultrafast idk

🚨 Big news today: We’ve raised $100M for our Series A! 🎉 🚀 We believe demand for fast inference is many times larger than demand for slow inference, and it’s about to go exponential. Thanks to our co-leads @BlackbirdVC and @Abstractvc. Two visionary partners join our journey 🎉 🚀

1:02
Brendan Graham avatar
⟳ Ani Rahul retweeted
Brendan Graham@brendanigraham·Oct 21

it turns out the implementation matters

The implementation no longer matters People really, really need to understand that

Post attachment
Eric Zhang avatar
⟳ Ani Rahul retweeted
Eric Zhang@ekzhang1·Oct 21

for the record, I really like DeepSeek’s benchmarking approach, it controls for all the free variables and is probably the most compelling recent roofline curve to me. every engine should be benchmarking this way, and kv caches/pd separately see:

Post attachment

If inferenceX didn't deprecate 8k/1k this wouldn't be a problem smh

Ani Rahul avatar
Ani Rahul@Ani_da_dev·Oct 21

To be fair they did say most of their stuff was based off vLLM I think the most interesting part was the kv cache offloading they mentioned but that implementation wasn’t shared

🚨🚨🚨 vLLM is all you need to reproduce DeepSeek’s performance. With Wide-EP and EPLB, vLLM can match DeepInfer’s decode performance within 6% and generate $39/gpu/hr in revenue at DeepSeek’s off-peak pricing, at 200 P90 TPS on the AgentX workload. Specifically: • Under DeepSeek’s decode configuration (DeepSeek-V4.1-Flash, 131K context, nextn=3), vLLM on GB200 achieves throughput within 6% of DeepInfer’s published results across batch sizes 8–256. • On AgentX, vLLM pushes the Pareto frontier further on GB300, reaching up to 1M tokens/s/GPU and generating $38.96/gpu/hr in revenue at 196 P90 TPS, based on DeepSeek’s off-peak API pricing. • Also, vLLM extended low-latency kernel support for DeepSeek-V4.1-Flash, achieving up to 826 TPS per user on AgentX.

Post attachment
wh avatar
⟳ Ani Rahul retweeted
wh@nrehiew_·Oct 20

Post attachment

🚀 Introducing DeepInfer, our in-house inference framework, now fully open source! Powering DeepSeek-V4.1-Flash at scale with high throughput and low latency. We’re sharing our production stack to help the community lower compute costs, reduce energy use, and make AI more affordable. 🔗 github.com/deepseek-ai/DeepI… 1/4