👋 I build with AI in the open, explore open models, and share what I learn.
Here’s what I’m working on:
LLM benchmarks & evals
⚡ ZebraBench: benchmark LLM inference right in your browser
sujee.github.io/zebrabench/
🧪 Practical LLM Evals: tools and examples for evaluating LLMs
github.com/sujee/practical-l…
Playful AI demos
🎨 Doodle Derby: you draw, vision models race to guess
sujee.github.io/doodle-derby…
🏇 LLM Derby: models race to answer trivia
sujee.github.io/llm-derby/
Visual explainers
🎬 Visual Stories: animated explanations of AI and software concepts
github.com/sujee/visual-stor…
Follow along as I build, experiment, and share what I learn.
🛠️ More projects: sujee.dev/projects/
🎤 Talks & workshops: sujee.dev/speaking/
Missed a CFP deadline by a day or two? It’s still worth checking. Sometimes the submission form is still open and organizers are accepting late submissions.
In CFP Tracker, the “Upcoming (> -10 days)” view includes upcoming deadlines and ones that passed in the last 10 days.
Love speaking at conferences but miss CFP deadlines? 🎤
I built CFP Tracker to find speaking opportunities and track my submissions in Obsidian.
It pulls calls for papers from multiple public sources and saves them as Markdown notes in my Obsidian vault.
In Obsidian, I can easily search for CFPs by topic, location, and deadline, shortlist conferences I’d like to speak at, and track my talk submissions.
Since it’s all local Markdown, I can also ask an agent, “Which AI conferences have CFP deadlines in the next 30 days?”
CFP Tracker is open source. If you give it a try, I’d love to hear where you find CFPs.
Watch the 30-second demo. Repo 👇
Here is a sample view of @obsdmd Bases for CFP Tracker.
I can filter by topic and location, sort by deadline, and shortlist conferences right from my vault. 🎤
Love speaking at conferences but miss CFP deadlines? 🎤
I built CFP Tracker to find speaking opportunities and track my submissions in Obsidian.
It pulls calls for papers from multiple public sources and saves them as Markdown notes in my Obsidian vault.
In Obsidian, I can easily search for CFPs by topic, location, and deadline, shortlist conferences I’d like to speak at, and track my talk submissions.
Since it’s all local Markdown, I can also ask an agent, “Which AI conferences have CFP deadlines in the next 30 days?”
CFP Tracker is open source. If you give it a try, I’d love to hear where you find CFPs.
Watch the 30-second demo. Repo 👇
Muse Ember is out. It is a small, efficient model for everyday tasks. I put it in LLM Derby with Spark and Wonder. 🏇
I built LLM Derby to measure two things in the same run: how accurately each model answers and how quickly its endpoint responds.
All three models get the same questions. Each correct answer moves the model’s horse one step. The first to 20 wins.
Ember takes an early lead on the Hard questions. Its median response time is 1.3 seconds. Wonder is in second place, with a median response time of 4.1 seconds.
About 20 seconds in, Ember gets its first Expert question and takes longer to answer. Wonder catches up.
Race ends in 2 mins. Watch the race 👇
Hard questions test factual knowledge.
- Which naturally occurring noble gas has the highest boiling point? (Radon)
- Which element’s name comes from the Greek word for “lazy” or “inactive”? (Argon)
LLM Derby generates new Expert questions for each race. These questions cover math, calendar calculations, and problems with multiple steps. The program calculates the correct answers.
- Multiply 80,966,550 by 92,349,694. Add the digits of the product. What is the sum? (66)
- How many days are there from June 20, 1841, to May 18, 1882? (14,942)
A few takeaways from this run
- Response time made a big difference. All three models got every Hard question they answered correct. Ember and Wonder each got 15 right. Spark got 12 right before the race ended.
- The first Expert question changed the pace. Ember’s early lead diminished while it worked on that question for longer.
Today we're introducing Muse Ember, our smallest and most efficient model, built for complex reasoning execution and everyday agent work.
Muse Ember can carry workflows across apps, call tools in sequence, run tests and make targeted code fixes. With a 1M-token context window and native understanding of text, images, audio and video, it can also work across large document sets and varied inputs.
I turned this little experiment into a blog post! ✍️
More details on the Sri Lankan Tamil vs. Tamil Nadu Tamil comparison, what I learned about Agaram 2.5B, and the interesting regional vocabulary differences.
Would love to hear your thoughts, especially from Tamil speakers!
sujee.dev/post/agaram-tamil-…
GPT-6 Astra rewriting Qwen3.5-0.8B s weight to play a better Breakout game!
one shot / no training / no tools!
just amazing!
See more examples in their research paper : rikiyat.github.io/blog/weigh…
We asked GPT-6 Astra to edit Qwen3.5-0.8B weights to improve math.
It directly changed only 0.016% of weights, fixing 0.8B's reasoning loops & improving math up to 6x and even instruction following 3x, without gradients or training.
We call this Weight-Space Programming: frontier LLMs can now natively read/write neural network weights! 🧵
Blog: rikiyat.github.io/blog/weigh…
@Mechramc Curious about what's next for Agaram! 😁
- More training, perhaps with broader coverage of regional Tamil dialects? 🙂
- System prompt support to customize the model's personality and behavior?
Excited to see where this project goes!
Curious where the KV cache comes into play?
In this video, I follow a single request from client to chip and back, including how the KV cache fits into inference.
What actually happens when you call an LLM API? 🤔
I've been working on this visual explainer for a while, and I'm excited to finally share it! 🎬
Follow a single request from client → gateway → tokenizer → router → queue → GPU (prefill + decode) → streamed response.
Along the way, explore optimizations like:
KV caching / Prompt caching
Cache-aware routing
Continuous batching
Disaggregated prefill/decode
Speculative decoding
I recently gave a talk on this at Hacktoberfest 2026, but here's the entire journey in just 5 minutes!
Watch it and let me know what you think! 👇
Full talk and repo in the reply.
What does it take to run Kimi K3, a massive 2.8-trillion-parameter model, for inference?
Enjoyed this whiteboard walkthrough by @seanlie and @MilksandMatcha from @Cerebras.
Love the warehouse/lab setting too :-)
Here’s my takeaway from their Kimi K3 example.
K3 has 93 layers:
• 69 KDA (Kimi Delta Attention) layers using linear attention
• 24 Gated MLA (Multi-head Latent Attention) layers using full attention with a compressed KV cache
The memory distinction matters: KDA maintains fixed-size state per request. MLA keeps a KV cache that grows with context length.
Now the math:
2.8T parameters at 4 bits each ≈ 1.4 TB of raw weights.
Following Cerebras’ example, let’s round that to 3T parameters:
• 3T × 4 bits ÷ 8 ≈ 1.5 TB
• 1.5 TB ÷ 93 layers ≈ 16 GB/layer on average
Each wafer has 44 GB of on-chip SRAM. Cerebras’ illustrative mapping is:
• Two linear-attention layers (16 GB each) per wafer: ~32 GB of weights, leaving ~12 GB
• One full-attention layer per wafer: ~16 GB of weights, leaving ~28 GB for KV cache and other state
That gives:
• 69 linear-attention layers ÷ 2, rounded up = 35 wafers
• 24 full-attention layers ÷ 1 = 24 wafers
• Total: 59 wafers
This example assumes 4-bit weights, an 8-bit KV cache, and 256K average request lengths.
The neat part: weights stay on-chip while activations move through the wafer pipeline.
Larger models? Like 5T or 10T? :-)
Extend the pipeline with more wafers. The exact count depends on the model architecture and memory requirements.
Highly recommend watching the full video!
How we deploy multi-trillion parameter models at scale
Today, let's break down the math for how @cerebras:
1. Fits the full model into on-chip memory
2. Maps the model across multiple chips
3. Handles a high # of concurrent requests
So I turned my curiosity into a little experiment! 🇱🇰🇮🇳
I put together 20 everyday phrases, each written in Sri Lankan Tamil and Tamil Nadu Tamil, with the same meaning.
Then I asked Agaram 2.5B to translate all 40 sentences into English.
Results:
🇱🇰 Sri Lankan Tamil: 12/20 ✅ (60%)
🇮🇳 Tamil Nadu Tamil: 17/20 ✅ (85%)
see screenshot
Some interesting regional vocabulary differences:
- வடிவா vs. அழகா (beautiful)
- கதைக்க vs. பேச (to talk)
- இருங்கோ vs. உக்காருங்க (sit down)
The model recognized quite a few Sri Lankan Tamil expressions!
My guess is that training data plays a role here. Tamil Nadu Tamil likely has more representation in the training corpus than Sri Lankan Tamil.
This is just a small, informal experiment, not a comprehensive benchmark. But it was fascinating to see how regional language variations affect model understanding!
Huge kudos to @Mechramc and @MuraiLabsHQ for building and open-sourcing a Tamil-first model! 👏 It's exciting to see more AI development focused on languages like Tamil.
I've shared my experiment, including the test data, Python code, and full results. Feel free to reproduce it, extend it, or share your feedback! 😁
github.com/sujee/ai-playgrou…
This one's for my Tamil-speaking friends!
Trying out Agaram 2.5B (அகரம்) - a Tamil-English language model trained from scratch by @MuraiLabsHQ & @Mechramc.
Running locally on my Mac with Ollama! 🦙
First, some casual chatting.
English:
>>> how are you?
> I'm doing well, thank you for asking!
Tanglish (Tamil written in English letters):
>>> eppadi suham?
> I'm doing great!
Tanglish, requesting a Tamil response:
>>> eppadi suham? answer in tamil
> நான் நல்லா இருக்கேன். உங்களுக்கு என்ன உதவி வேணும்னு சொல்லுங்க, நான் பண்றேன்.
Even got it to write a poem about cats! 🐈
>>> write me a poem about cats
> நிச்சயமாக! இதோ ஒரு குட்டி கவிதை:
> வீட்டின் காவலன்,
> வெயிலில் உழைக்கும் நண்பன்,
> அன்பான நண்பன்,
> என் செல்லக்குட்டி!
Pretty cool for a 2.5B model!
But then I got curious: How well does it understand different Tamil dialects, particularly Sri Lankan Tamil vs. Tamil Nadu Tamil?
One question led to another, and before I knew it, I was down a rabbit hole testing Tamil dialects! 🐇
Here's what I found. 🧵👇
Agaram 2.5B is out. It's a Tamil-first language model we trained from scratch at @MuraiLabsHQ , our small non-profit lab. As far as we know, it's the first Tamil model trained from scratch that can call tools.
It chats in Tamil, Tanglish and English, translates, and can call tools. We're releasing the base model, chat model and GGUF files under Apache-2.0. You can download it and run it offline on your own laptop.
The first chat model had a four-day public preview. Seeing where it fell short with real users helped us decide what to work on. We rebuilt the fine-tuning around what we saw, using 222,662 records.
Thank you to everyone who spent time with that early version.
The video walks through the model. You can try it here:
hf.co/spaces/mechramc/agaram…
நன்றி 🙏