←
erogol8,734 posts
erogol avatar

erogol

@erogol

Doing ML Web - erogol.com Substack - erogol.substack.com

◷ Joined October 2008▧ 360 Photos and videos
erogol avatar
erogol@erogol·16 Jul 2025

XTTS is still being downloaded almost 5m times every month and 2.1m only on HF. It is greater than many recent hyped models. Hope people use it well for that to worth to my burnout that I’m still recovering from. Coqui has been one of the most successful broke startups

Post attachment
erogol avatar
erogol@erogol·Nov 17

A 4-min explainer on "Weight-Space Programming": • MIT's Sparks of WSP (GPT-6 Astra) • @roicohen9: one paragraph in, a Qwen weight update out • @HyunjongOk: sparse tables, ~1,800× fewer params (Narrated by Phonon 31M )

4:19
Joey Turnbull avatar
⟳ erogol retweeted
Joey Turnbull@joey__turnbull·Nov 16

1/ We still don’t have a simple picture of how even 2-layer MLPs learn features. In a new paper with @berkott3 and @learning_mech, we propose a simple, solvable model of shallow MLP training, and find that real MLPs roughly follow it. 🧵

Post attachment
Brandon Amos avatar
⟳ erogol retweeted
Brandon Amos@brandondamos·Nov 16

⚠️ Numerical issues can go on silently for decades like in the softmax/CE! Similar issues are present in other widely-used divergences like the KL and JSD I just posted this paper overviewing what they are and how to fix them: 📚 arxiv.org/abs/2611.18511

Post attachment

Hi all, I have a new result I'm excited to share. It turns out that *all existing implementations* of softmax and cross-entropy (that's JAX, PyTorch, textbooks, etc) have a subtle bug --- two bugs, actually. A simple rewrite fixes both, and eliminates "slingshots" when grokking:

0:07
Hyunjong Ok avatar
⟳ erogol retweeted
Hyunjong Ok@HyunjongOk·Nov 17

🚀 Stop using Transformers for weight-space programming If you are not training with gradients on a GPU, why default to a Transformer? When an LLM writes network weights directly, letting it also design the architecture achieves: 📉 ~1,800× fewer parameters at matched accuracy on ARC (AI2 Reasoning Challenge), compared to the best Transformer the same LLM wrote. ⚡ Up to 17× faster GPU inference with a memory architecture built for these tables, compared to a Transformer of the same size. 🧵 Paper & code at the end 👇

Post attachment
erogol avatar
erogol@erogol·Nov 16

Added tiny-voca to my small CPU TTS benchmark. 4.5M params, Apache-2.0. On 1 CPU thread: - 117 ms to first audio - 37x real time - 3.3% WER - 666 MB memory 2nd fastest of 13 models. Phonon 31M stays my pick for live voice. Hear all: erogol.com/small_tts_bench/

Nazeem@n4ze3mNov 15

I trained a text-to-speech model that has never heard a real human voice. 4.5M parameters. 18 MB. Runs on a CPU. Meet tiny-voca

0:28
erogol avatar
erogol@erogol·Nov 13

Small CPU TTS Benchmark 12 tiny TTS models on 1 CPU thread, 60 audio samples. My Pick -> Phonon 31M @GradiumAI. 82 ms to ttfa, 4x rtf, 1.6% WER, 363 MB, Apache-2.0. Pocket TTS, same box: 286 ms, 1.2x, 1347 MB. Let me know any models i missed erogol.com/small_tts_bench/

Post attachment
erogol avatar
erogol@erogol·Nov 13

Here is a short video about "Decode Before You Hear" paper that speeds up chunk based speech to speech models significantly arxiv.org/abs/2611.06982

3:26
erogol avatar
erogol@erogol·Nov 12

Machine Learns - 81 🤖 • Weight-Space Programming: a closed LLM edits raw weights • WaveGen: TTS with no codec or vocoder • Decode Before You Hear • Latent Sparse Attention • Agentic Backdoors • Mistral Large 4 • My small CPU TTS benchmark erogol.substack.com/p/machin…

Post attachment
erogol avatar
erogol@erogol·Nov 11

New best val loss in BlaGPT (3.1106): - Aurora optimizer beat Muon (-0.0294 loss) - RoPE + RoVE beat GRAPE-A with 0 extra params & faster steps - GatedNorm + Engram x20 stacked cleanly Code & logs: github.com/erogol/BlaGPT

erogol avatar
erogol@erogol·Nov 7

The Kimi K3.1 Pro is a beast! It’s enough for 99.9% of AI work in a company and there’s no reason to pay premium for Claude or OAI. I’ve tried to find any shortcomings but I can’t seem to find any.

erogol avatar
erogol@erogol·Nov 6

Gemini iOS app is embarrassingly bad. I tried to use it after a long time. Got 3 connection errors after a long wait Now I reached my limit and can’t ask anything

erogol avatar
erogol@erogol·Nov 6

I don’t know why we need computer use It is like using a robot to teach on a black board

In your experience, Computer use works best: - on mac OS? - on Windows? - on Linux?