←
Raunak2,063 posts
Raunak avatar

Raunak

@raunakdoesdev

I’m into open science, elegant software architecture, and well-crafted interfaces. Building reducto.ai

◷ Joined July 2017▧ 217 Photos and videos
Raunak avatar
Raunak@raunakdoesdev·20h

Last week, I said every parser is #1 on its own benchmark. So we tested the benchmarks. We ran 8 parsers across 12 benchmarks, then tilted the pages by 2–4°. 6 benchmarks got a different winner. We’re calling it BenchBench.

Post attachment
Raunak avatar
Raunak@raunakdoesdev·20h

On 9 of 12 benchmarks, the median percentage score drop after tilt was larger than the percentage gap between first and second. The scorer made a difference too. One ParseBench-2 table kept every number but scored zero because of how the cells were arranged.

Post attachment
Raunak avatar
Raunak@raunakdoesdev·20h

The content didn’t change. We kept each benchmark’s own scoring rules (adjusting golden bounding boxes for tilt where applicable). 5 of 8 providers moved in the overall average-rank comparison.

Post attachment
Raunak avatar
Raunak@raunakdoesdev·Nov 15

mini rant on OIDC and workflow identity federation tldr: reducto supports it, you should avoid using static credentials wherever possible, and toss your api keys. more here: reducto.ai/blog/workload-ide…

1:38
Luke Igel avatar
⟳ Raunak retweeted
Luke Igel@lukeigel·Nov 14

The @trykino launch video is now open source. Remix its footage and code at the link below. We're releasing all our project code (all edits and graphics in Kino are code!), raw footage, odd VFX ideas, and comments from our many Granola calls

0:37

Introducing Kino: make videos end-to-end with Claude Code and Codex. Drop your footage, get highlights, edit together, send links for feedback, and let your agents cook. AI is becoming a great creative partner. Kino is a new home for your team and agents to collaborate.

1:35
Raunak avatar
Raunak@raunakdoesdev·Nov 12

This benchmark came out a few hours ago. We were able to audit this quickly because modern frontier LLMs are great at comparing parsers. There is no reason to be using rule-based benches anymore, LLM judge is the way. parsejudge.com/parsebench-2

Adit@aditabrmNov 12

We took a look at ParseBench-2 today. On their benchmark, one table scored 0 and another scored 100. Can you tell which is which? 🧵

Post attachment
Adit avatar
⟳ Raunak retweeted
Adit@aditabrm·Nov 12

We took a look at ParseBench-2 today. On their benchmark, one table scored 0 and another scored 100. Can you tell which is which? 🧵

Post attachment
Fardeem avatar
⟳ Raunak retweeted
Fardeem@FardeemM·Nov 11

in approximately 2 weeks everyone on the TL will be talking about how insane devin [redacted] is Poke team really cooked

Reducto avatar
⟳ Raunak retweeted
Reducto@reductoai·Nov 11

You can now go from your local terminal to an editable document in 30 seconds ⚡ Connect Reducto Office to your Pi agent via MCP, ask for a launch brief, and open it in the editor to keep working together.

0:31
Hunter Leath avatar
⟳ Raunak retweeted
Hunter Leath@jhleath·Nov 11

in my opinion, this is actually the correct way to do benchmarking. nothing beats the user’s own code. what would folks like to see if we did this for Archil?

Every document parser is #1 on its own benchmark. So forget static benchmarks. Today, we're releasing ParseJudge, an open-source app that judges parsers on your own documents, free for up to 10,000 pages a day. parsejudge.com

2:03
Raunak avatar
Raunak@raunakdoesdev·Nov 10

see if you can spot the easter eggs in this video :) there are ~3 good ones. most of my team hasn't spotted them yet 😛

Raunak avatar
Raunak@raunakdoesdev·Nov 10

fun fact: I scripted, filmed (on my iPhone), and edited this video all in a 5 hour span on Sunday with no video experience! impossible without @trykino

Every document parser is #1 on its own benchmark. So forget static benchmarks. Today, we're releasing ParseJudge, an open-source app that judges parsers on your own documents, free for up to 10,000 pages a day. parsejudge.com

2:03