←
Avyay Varadarajan126 posts
Avyay Varadarajan avatar

Avyay Varadarajan

@avyvar

co-founder @daridotdev // prev @ycombinator, @caltech, basketball data @lakers, software @uber, startups

◷ Joined April 2018▧ 23 Photos and videos
Avyay Varadarajan avatar
Avyay Varadarajan@avyvar·Jul 24

Today, we're releasing our open-weight, auto-routing model @daridotdev, built for coding agents. We're state-of-the-art on the Pareto Frontier, w/ 70% cost reduction + comparable coding performance to Fable. Bring your own evals, choose your models, or use our defaults.

Post attachment
J Yim avatar
⟳ Avyay Varadarajan retweeted
J Yim@mwahjiwoo·Nov 11

Introducing Rollup. Your app never has to ask a new user who they are again. Their camera roll already knows, and your app should too. Rollup is the first SDK that turns years of photos into user context, opt-in and fully on-device. Add to your app👇 tryrollup.com

0:42
Avyay Varadarajan avatar
Avyay Varadarajan@avyvar·Nov 10

For small teams, subs are the best way to save on tokens. Route based on how much usage is left, cache warmth + automatically claim resets. Run subscription pooling fully locally, or use free hosted to get detailed analytics across your team. github.com/mupt-ai/dari

Avyay Varadarajan avatar
Avyay Varadarajan@avyvar·Nov 10

They spent millions on router dot com. We spent $20 on routerer.com. We saved money on the domain to save you money on tokens. > Pool Claude + Codex subs > Pick the right model + thinking level w/ our SOTA router > Use your traces to fix token waste > Open-source

1:35
Avyay Varadarajan avatar
Avyay Varadarajan@avyvar·Oct 23

Are OpenAI models not the same on sub / API? Ran GPT-6 Luna (high) on DeepSWE 1.1 in Codex: > With subscription: 26% solved > With API (via Vercel AI Gateway): 41% solved Thought it might be variance, but the sign-in run also used ~4x fewer reasoning tokens per task

Avyay Varadarajan avatar
Avyay Varadarajan@avyvar·Oct 17

DeepSeek-V4.1-Flash-1017 is Pareto-mogging in our initial tests. It's the first open-source model we've tested that's consistently on the Pareto frontier in real-world PRs, running in @pidotdev Some results from Sentry, PostHog, and Next.js PRs:

0:09

🚀 DeepSeek-V4.1-Flash-1017 is here! Stronger benchmark performance, faster task completion, and shorter outputs than the previous version—a Pareto improvement in performance and cost. Now live on our website, app, and API. 🛠️ Optimized for @DeepSeekHarness, Tencent CodeBuddy, WorkBuddy, and more. Upgrade to DeepSeek Harness v0.2.1-rc.1 or later for the best experience with the new model. 1/4

Post attachmentPost attachment
Avyay Varadarajan avatar
Avyay Varadarajan@avyvar·Oct 7

Haiku 5.5 is an incredibly capable model at its size. We ran Haiku on real Pi, Next.js, PostHog, and Sentry tasks, and it was on the Pareto frontier in every test. Slightly higher $/task than Luna for increased performance. Towards intelligence too cheap to meter!

0:12

Open-weight models may be costing you more on real work. We built self-bench to turn real GitHub PRs into evals. Testing repos like Next.js, PostHog, and Sentry, open models were usually pricier + less capable. Find the best model for your repo: selfbench.dev

0:40
Avyay Varadarajan avatar
Avyay Varadarajan@avyvar·Oct 7

Is @MistralAI Large 4 smart or just chonk-y? We ran Le Chonk on real PRs from Next.js, Sentry, Pi, and PostHog. > Mistral Large 4 generally benchmarked worse + similar price as Kimi K3 and GLM 5.3 in the Pi harness. > None of the 3 models were ever on the Pareto frontier.

Post attachment

Open-weight models may be costing you more on real work. We built self-bench to turn real GitHub PRs into evals. Testing repos like Next.js, PostHog, and Sentry, open models were usually pricier + less capable. Find the best model for your repo: selfbench.dev

0:40
Techmeme avatar
⟳ Avyay Varadarajan retweeted
Techmeme@Techmeme·Oct 5

Survey: only 11% of ~400 businesses could forecast AI spending; a study finds lower-priced models cost more than higher-priced models on 32% of 6,800+ tasks (Wall Street Journal) (Visit Techmeme dot com for the link and full context!)

Avyay Varadarajan avatar
Avyay Varadarajan@avyvar·Oct 5

On our eval of 82 tasks from real @sentry PRs, GLM 5.3 was > 50% more expensive than claude opus 5.5 ($1.83 vs $1.22 per task) > 8.5 points less accurate (74.4% vs 82.9%)

0:34

Open-weight models may be costing you more on real work. We built self-bench to turn real GitHub PRs into evals. Testing repos like Next.js, PostHog, and Sentry, open models were usually pricier + less capable. Find the best model for your repo: selfbench.dev

0:40
Avyay Varadarajan avatar
Avyay Varadarajan@avyvar·Oct 5

Per-token costs don't mean the model is actually cheaper for your work. In almost all of our tests, "cheaper" per-token models like GLM 5.3 and Kimi K3 were *more* expensive than models like GPT 6.1 Sol per task, due to token efficiency.

Avyay Varadarajan avatar
Avyay Varadarajan@avyvar·Oct 5

With benchmaxxing, it’s become hard to trust public evals. When a new eval is released, model providers rush to saturate it. This doesn’t mean a model actually performs well in your actual work. Through iterative agent review, self-bench builds quality evals you can review.

0:46