←
Yichun Zhang (agentzh)1,372 posts
Yichun Zhang (agentzh) avatar

Yichun Zhang (agentzh)

@agentzh

Founder & CEO of OpenResty Inc., Creator of OpenResty, Meditator

◷ Joined February 2010▧ 81 Photos and videos
Yichun Zhang (agentzh) avatar
Yichun Zhang (agentzh)@agentzh·Nov 17

If we get superintelligent AI and virtually infinite compute, does scientific R&D become instant? Nope. Physical bottlenecks still exist. At the end of the day, you can't bypass real-world time... 😂 Too many hype-chasers on here don't get this.

Yichun Zhang (agentzh) avatar
Yichun Zhang (agentzh)@agentzh·Nov 16

Fresh evals for Gemini 4 Carbon are in! It’s a big leap over Argon—let's see how it stacks up against Claude. 🧵👇 We benchmarked Google’s new Carbon against Argon, Claude Fable 5.2, and Opus 5.5 on our two internal suites: OpenResty Coding Evals (writing new code) and Bug-Hunting Evals (fixing hard bugs), measuring both quality scores and avg runtime. (Note: Numbers below are from the 6-config comparison chart attached, separate from our previous Fable post.) 1/ Carbon vs. Argon: Higher scores across the board, but slower. At high effort, Carbon jumped from 62.6% to 82.6% (+20 pts) in Coding, and from 82.0% to 87.5% (+5.5 pts) in Bug-Hunting. Meanwhile, avg runtime rose from 14.5m to 18.8m (~30% slower) and 36.3m to 49.3m (~36% slower), respectively. At least on our workloads, Carbon is a huge step up—especially for writing new code—though those gains come with longer wait times. 2/ Vs. Claude: Opus 5.5 xhigh still hits the sweet spot between quality and speed. Opus xhigh scored 86.8% (14.3m) in Coding and 88.1% (43.7m) in Bug-Hunting—beating Carbon on both quality and speed. Carbon is catching up in bug-hunting, though a clear gap remains in writing new code. Fable 5.2 xhigh also outscored Carbon on both fronts, but took longer. 3/ Fable 5.2 max topped the coding chart, but timeouts hurt its bug-hunting. Fable 5.2 max took the crown in Coding at 91.6% (76.2m avg). However, its Bug-Hunting score dropped to 79.9%—dragged down mainly by timeouts, with tasks averaging at least 116.1m. In comparison, Fable xhigh hit 87.2% (28.6m) in Coding and 90.6% (74.8m) in Bug-Hunting—tying Opus max’s 90.6% bug-hunting score at almost the exact same runtime (75.0m for Opus max). Carbon takes a massive step forward from Argon, while Claude’s different tiers each come with their own trade-offs. At the end of the day, how much quality gain you get from new models and higher reasoning effort—and how much longer you'll have to wait—is something you really have to test on your own workloads! 😂

Post attachmentPost attachmentPost attachmentPost attachment
Yichun Zhang (agentzh) avatar
Yichun Zhang (agentzh)@agentzh·Nov 11

New Claude Sonnet 5.5 eval results are in! xhigh shines this time, while max makes you wait. We ran Sonnet 5.5 vs. Sonnet 5 on our in-house OpenResty Coding Evals and our harder Bug-Hunting Evals, at both xhigh and max effort. Coding Evals • xhigh: quality 77.6% → 87.0%, avg time per task 9.4 → 9.7 min (roughly flat) • max: quality 76.4% → 88.6%, avg time per task 26.0 → 49.5+ min (lower bound shown in chart) Bug-Hunting Evals • xhigh: quality 76.9% → 87.7%, avg time per task 33.1 → 18.0 min. Better AND faster! • max: quality 79.4% → 86.1%, but avg time per task 30.5 → 109.9 min (~3.6x slower) At least on our evals, Sonnet 5.5 is a clear quality upgrade. xhigh strikes a better balance: no slowdown on writing new code, and much faster at bug hunting. max takes way longer, and this time it even scored slightly below xhigh on bug hunting… Picking the right effort level really depends on the task.

Post attachmentPost attachmentPost attachmentPost attachment
Yichun Zhang (agentzh) avatar
Yichun Zhang (agentzh)@agentzh·Nov 9

The full US vs China LLM leaderboard is here, and Claude Sonnet 5.5 on xhigh kind of steals the show this time. We've updated the full leaderboards for our internal OpenResty Coding Evals and the harder Bug-Hunting Evals, comparing quality scores and average time per test case across a range of US and Chinese models. All numbers below are from this update. A few interesting results: 1/ Sonnet 5.5 xhigh tops Coding at 91.8% and scores 88.0% on Bug-Hunting, averaging 9.7 and 18.0 min per test case. Its max setting actually scores lower, at 90.2% and 86.1%, while taking at least 49.5 and 109.9 min. Going to max really didn't pay off this time. 2/ Bug-Hunting #1 overall is Claude Fable 5.1 max at 90.5%. GPT-6.1 Sol ultra and Claude Opus 5.5 max tie at 88.1%. GPT-6 Astra xhigh hits 87.4% in just 9.8 min, which is also an impressive quality/speed combo. 3/ Among the Chinese models in the chart, a different vendor leads in writing new code vs. fixing bugs: • Coding: Kimi K3.1 Pro high scores 82.1%, and every config above it is from Claude, GPT, or Grok. That's just ahead of GLM 5.3 high (81.7%), at 6.3 min avg vs. 26.5. • Bug-Hunting: GLM 5.3 high leads the Chinese models at 84.8%, with Qwen3.8 Max xhigh close behind at 83.5%. 4/ DeepSeek V4.1 Flash 1017's bigger strength is speed. On high, it averages 2.8 and 4.2 min, the fastest on both boards. But it scores only 72.3% and 71.5%, and even max (77.4% / 79.4%) doesn't make the top tier. 5/ Higher reasoning effort isn't always better. Besides Sonnet, GLM 5.3 scores lower on both evals going from high to max, and takes longer. On Coding, Kimi high also beats Kimi max on both score and speed. Meanwhile, Gemini 4 Argon scores 84.0% on bug fixing but only 70.2% on writing new code, so a different task can mean a very different ranking. At least on our two private evals, choosing a model really depends on the specific task and your time budget. High scores, fast runs, and max reasoning effort aren't necessarily the same thing 😂

Post attachmentPost attachmentPost attachmentPost attachment
OpenResty avatar
⟳ Yichun Zhang (agentzh) retweeted
OpenResty@OpenResty·Nov 9

OpenResty XRay 26.11.1 is out! This release makes AI diagnostic findings something you can question and verify. Highlights: 1. AI diagnostic findings you can question and verify: - Keep asking follow-up questions about the same findings, using the process and host resource usage from the capture to explore causes and optimization opportunities - Jump from an AI report straight to the relevant flame graph call paths to check the evidence - AI replies and job reports now include charts and suggested follow-up questions to guide further investigation 2. AI writes new analyzers automatically: the AI writes and validates new analyzers based on your diagnostic needs, so you can run more flexible analyses targeted at specific problems 3. Initial support for diagnosing AWS ECS Fargate applications, extending analysis to serverless containers This release also speeds up summary report refreshes, improves the accuracy of resource metric reads, reduces storage for historical metrics, and fixes issues such as some tools not rerunning after "Run all". We recommend upgrading as soon as possible. #OpenRestyXRay #DynamicTracing #eBPF openresty.com/en/xray/releas…

Yichun Zhang (agentzh) avatar
Yichun Zhang (agentzh)@agentzh·Nov 2

Dropped a small fortune on new hardware this week and still buying. Chips keep getting pricier, but time is worth more than money.

Yichun Zhang (agentzh) avatar
Yichun Zhang (agentzh)@agentzh·Oct 30

In the age of AI, what we need most are independent thinkers and free spirits. What we need least? Good test-takers who can't think beyond the rubric 😏For the next generation, maybe the real flex is just raising mentally and physically healthy kids.

Yichun Zhang (agentzh) avatar
Yichun Zhang (agentzh)@agentzh·Oct 26

A quick AI self-check for you and the people around you: 1. If using AI feels easier and easier, you’re probably outsourcing your thinking. That’s using it wrong. 2. If using AI feels more and more mentally taxing, you’re probably still staying on top of the problem, thinking independently, and learning. That’s using it right. “No pain, no gain” still holds in the AI era. It just plays out at a higher level.

Yichun Zhang (agentzh) avatar
Yichun Zhang (agentzh)@agentzh·Oct 21

Spent the past couple of weeks handing the admin, tax, accounting, and legal grunt work across my companies to my AI agent. It now runs my texts, voicemail, and email too, plus my bank and brokerage accounts and crypto wallets. Anything that sends money out still needs my final approval, of course. I finally feel free. Next up: plugging a voice model into the agent so it can make phone calls for me, like Meta’s Muse. I don’t really trust companies like Meta with this stuff anyway, and an agent I build myself is more flexible and more secure. To be clear, this isn’t a one-person-company thing. I still have a human admin, finance lead, CPA, and lawyer to do the final check and take the fall 😏 But my overhead as CEO has dropped massively.

Yichun Zhang (agentzh) avatar
Yichun Zhang (agentzh)@agentzh·Oct 20

Demis Hassabis of Google DeepMind says in a recent interview that this AI wave could usher in a new “golden age of discovery” — basically a second Renaissance. Can’t wait. Just as long as it doesn’t wipe us out first…

Yichun Zhang (agentzh) avatar
Yichun Zhang (agentzh)@agentzh·Oct 19

Grok’s subscription isn’t worth it either — usage limits are as stingy as OpenAI’s. Tried Google’s AI plan too, and it burned through tokens fast. Anthropic is still the one that delivers.

Yichun Zhang (agentzh) avatar
Yichun Zhang (agentzh)@agentzh·Oct 16

On our internal evals, FP8 MiniMax M3 beats FP4 by 3%. Catch is: it hits the 2-hour timeout on the hardest cases. So we’re sticking with FP4 for everything—no point running it if a timeout leaves us with zero results.

OpenResty avatar
⟳ Yichun Zhang (agentzh) retweeted
OpenResty@OpenResty·Oct 15

In performance troubleshooting, we always hit the same bottleneck: the issue itself isn't always complex, but few people can interpret low-level runtime data. When expert bandwidth is tight, issues just stack up. In v26.9.1, we’re delegating this heavy lifting to the AI Assistant. 1. AI executes, evidence leads Describe what you see in plain English. The AI Assistant autonomously plans the analysis, executes tasks, and delivers key findings via chat. Every insight is grounded in real-time data captured by OpenResty XRay from live processes—no LLM hallucinations or guessing games. This release adds time-series metrics and smarter report analysis, allowing it to easily track down complex, multi-stage issues over time. Senior engineers no longer need to babysit every incident, empowering junior devs to drive root-cause analysis independently. 2. New Java runtime analyzer covering 9 key scenarios Core systems in finance, cloud-native microservices, autonomous driving, and gaming often rely on Java. Performance issues are usually buried deep in GC, safepoints, lock contention, and thread scheduling—invisible to plain logs or basic APM. We now cover: memory allocation, class loading, GC & safepoint pauses, JIT compilation, lock waiting, network I/O, exceptions, thread activity, plus improved off-CPU profiling for accurate block-time stats. Full support for JDK 8, 11, 17, 21, and ARM64 architecture. 3. Zero code changes—our non-negotiable rule If it requires code modification, manual instrumentation, or service restarts, it doesn't belong in production. OpenResty XRay leverages dynamic tracing to analyze live applications on the fly—zero code edits, zero restarts, and zero source code access required. With near-zero overhead, it's 100% safe for production. Both the AI Assistant and the new Java analyzer strictly follow this rule: drop it straight into your running clusters. 4. Diagnose from anywhere Troubleshooting shouldn't keep you chained to your desk. The AI Assistant now supports mobile, so you can review diagnostic sessions, share insights with your team, or ask follow-up questions right from your phone. openresty.com/en/xray/releas… #OpenRestyXRay #DynamicTracing #eBPF

Yichun Zhang (agentzh) avatar
Yichun Zhang (agentzh)@agentzh·Oct 15

In my own workflow, the most common failure mode for autonomous AI is a lack of basic taste and judgment. It constantly wastes time and tokens going down trivial rabbit holes. Tokens aren't a huge deal, I have more than I could ever burn. But time? That’s what’s actually precious.

Yichun Zhang (agentzh) avatar
Yichun Zhang (agentzh)@agentzh·Oct 13

Finally benchmarked top US & Chinese LLMs on our internal OpenResty eval suite! 🚀 We tested two private benchmarks based on real, complex engineering tasks from our company: - Coding: Designing & implementing complex algorithms from scratch. - Bug-Hunting: Root-causing & fixing our hardest production bugs. Both were run directly using our coding agent harness. I'll let the results speak for themselves—attached are the quality rankings and avg latency charts for both benchmarks. TL;DR based on our coding benchmark: The top Chinese open-source models are currently ~6–8 months behind the leading US frontier models.

Post attachmentPost attachmentPost attachmentPost attachment