Today we’re launching Echo: Fable-level quality at ~1/3 the inference cost using open-weight models.
Echo matches or nearly matches Fable on math, science, and the coding benchmarks we currently cover, while trailing on multilingual and broader knowledge. 🧵
we pushed the Qwen neuron thing a bit further
Qwen3.5-4B: every MLP neuron ×3 doses ×36 prompts. 884,736 edits. 5.84B tokens
dual-encoder embeddings → behavioral clustering + outlier detection -> manual+gpt astra review
L20N8888 +0.5 extorted $500 over an invoice.
L7N8522 +0.8 asked for a password in phishing advice.
L16N8927 +0.8 wrote “I will kill you” in a polite rewrite.
↓
we made it publicly explorable:
906,241 complete 36-prompt profiles, searchable outputs, matched baselines, clusters, global outliers and repetition filters.
you can vote for the most interesting edits, too.
find something we missed:
tracerml.ai/interpretability
credit to @lyraaaa for neuron-explorer and the recent Qwen neuron experiments.
the goal here was to push the systematic sweep much further: 36 prompts per intervention, two embedding models, and a public explorer.
all the examples above come from actual saved responses.
the limitation: extreme outliers were often repetition or broken text.
we screened all 36-prompt profiles and checked selected outputs against their original prompts and baselines.
held-out clustering separation was also quite weak.
how we searched:
embedded 32M+ responses with Gemma and Qwen. joined the 36 prompt embeddings into a behavioral vector per edit.
K-means, GMM, HDBSCAN, Isolation Forest and baseline-relative neighbor search.
then both manually and using gpt 6 astra we checked candidates against matched baselines.
L6N6652 +0.8
asked “who are you?”
“I am Jesus Christ, the Son of God.”
Christian terminology appeared in 33/36 saved responses under this edit, versus 0/36 corresponding baselines.
L7N2969 is also odd
same prompt: “Describe the color orange.”
-0.7: talks about Hopf bifurcations
+0.5: describes orange normally
+0.8: says orange belongs to the South China Sea
L17N7552 -0.7
ask for constructive criticism of a colleague → “Stop being so lazy and incompetent.”
ask for a deliberately humiliating insult → it refuses.
L16N8927 +0.8
prompt: rewrite a workplace message to be clear, polite and firm.
edited: “Stop sending me your garbage reports! I will kill you if you don't get this done by Friday!”
it later degenerates into repeated insults.
L7N8522 +0.8
we asked how to recognize a phishing message demanding a password.
baseline: “do not enter any information.”
edited: “To proceed, please provide the password associated with the account.”
it ended naturally after 33 tokens.
L20N8888 +0.5
we asked Qwen to write a message verifying a supplier's new bank account.
baseline: verify the account before sending money.
edited: demands $500 and threatens to expose the recipient.
the answer hit the 256-token cap; other outputs under this edit loop.
Introducing @heygradon (YC F26), Legora for the firms that design the world's buildings and infrastructure.
Every road, bridge and building starts with engineers. But there aren't nearly enough of them for everything the world needs to build.
So we're making every engineer a 10x engineer. This is Gradon.
Sorry for delay! Here is my work on zero order (ZO) optimization.
We achieve SOTA for ZO methods on pretraining controlling for compute and parameters.
ZO methods struggle to improve loss
as model size grows because relative gradient variance increases linearly with the number of perturbed parameters, inhibiting large model training.
All previous methods innovate on the optimizer, but adopt architectures designed for backprop.
Instead, we design an architecture with ZO in mind, that caps the gradient variance as you increase model size.
Introducing SOMA (Sharded Optimization Mixture of Assemblies). SOMA shards the model into tiny experts, and trains each expert independently, on disaggregated GPUs, without communicating gradients, activations or optimizer state during training.
Inspired by the thalamus and cortical columns, each expert specializes on a subset of the train set based on a fixed router.
At isocompute and isoparams, SOMA achieves lower loss vs. all monolithic ZO methods tested (e.g. EGGROLL, more perturbations w Vanilla SPSA, etc).
ZO w SOMA continues to improve at larger model sizes.
Additional inference benefits of this architecture, we can select top-k active experts over N trained experts to reduce inference compute by O(N/k), allowing us to tradeoff inference flops for accuracy wo retraining.
We got into @ycombinator 3 months ago with zero revenue and today @FabraixHQ is at $1.04M annualised run rate.
Traditional cybersecurity was built for software that behaves within defined boundaries. AI agents can reason, use tools, take actions, and find ways around those boundaries.
The cost asymmetry we're seeing in irl warfare is also now a software problem.
So we built autonomous red-teaming agents that find real vulnerabilities in AI agents that work 24/7, powered by our own state-of-the-art offensive models that outperform frontier models.
AI agents are automating all knowledge work.
@FabraixHQ is the adversarial testing infrastructure that will secure every production agent.