←
Vijay Sathish357 posts
Vijay Sathish avatar

Vijay Sathish

@Vslowsprinter

AI/ML eng @blockinc @stanford @nvidia studying how brains solve compute and learning. distilling books like 'principles of neural design' into ML takeaways.

◷ Joined February 2013▧ 15 Photos and videos
Vijay Sathish avatar
Vijay Sathish@Vslowsprinter·Nov 3

Wow, imagine a future where you could acquire new skills simply be changing neuronal connections or weights directly!

We asked GPT-6 Astra to edit Qwen3.5-0.8B weights to improve math. It directly changed only 0.016% of weights, fixing 0.8B's reasoning loops & improving math up to 6x and even instruction following 3x, without gradients or training. We call this Weight-Space Programming: frontier LLMs can now natively read/write neural network weights! 🧵 Blog: rikiyat.github.io/blog/weigh…

Post attachment
Vijay Sathish avatar
Vijay Sathish@Vslowsprinter·Oct 26

The deep dive article here:

Vijay Sathish @Vslowsprinter Oct 26 Article The Evolutionary 'algorithm' favors diversity over extreme efficiency To an ML expert’s eye, evolution looks like a vastly inefficient optimization process. However, evolution, operating at vast parallel scale, combined with vastness of time to accumulate and compound

Vijay Sathish avatar
Vijay Sathish@Vslowsprinter·Oct 22

The spike gets the spotlight. The pump pays the bill. Neuron signaling is 'free' because its like discharging a battery. The brain's energy (ATP) bill comes from restoring the charge. The life lesson: the real work always happens behind the scenes, away from the spotlight. Follow @Vslowsprinter for unique AI x bio takes and deep dives:

Post attachment
Vijay Sathish avatar
Vijay Sathish@Vslowsprinter·Oct 22

This is age old principle that is going to propagate up the entire stack - when constraints start to matter - chips, models, routers, data, agents, tools, apps and everything in between get specialized and the system complexity increases.

a16z@a16zOct 21

In the '90s, general-purpose chips wiped out the specialists. The original Thinking Machines, Cray Computer, and a wave of others shut down, and their best ideas got absorbed into other chips. Zain Asgar of @gimletlabs argues the conditions that made generalists win have changed. Dennard scaling (shrinking transistors made chips faster at the same power) ended around 2005, transistor costs stopped falling around 2012, and memory bandwidth never kept up with compute. Meanwhile, token demand is projected to grow 24x by 2030 while available power grows about 2x. When power is the bottleneck, performance per watt wins, and that's where specialized chips shine. Full piece from Zain on why "the time of specialization is upon us": a16z.news/p/why-hardware-spe…

Vijay Sathish avatar
Vijay Sathish@Vslowsprinter·Oct 20

8/9 Google’s recently demoed using reasoning to generate training experience (i.e. policy distillation). The bigger gap, in my view, is making this loop continuous and online. Can a robot turn effortful deliberation and repeated execution into reliable, reusable skills?

Vijay Sathish avatar
Vijay Sathish@Vslowsprinter·Oct 20

7/9 Robotics also mirrors this hierarchical control In Gemini Robotics, the Embodied Reasoning model plans, reasons and breaks complex goals into tasks. The Vision Language Action model senses and executes them, while the reasoning model monitors progress. The hierarchy exists.

Vijay Sathish avatar
Vijay Sathish@Vslowsprinter·Oct 20

6/9 The cerebellum predicts the consequences of actions. Comparing predictions with sensory feedback generates an error signal that is used to adjust motor system in real time. Fine control doesn’t require replanning the entire task.

Vijay Sathish avatar
Vijay Sathish@Vslowsprinter·Oct 20

5/9 But repeatable skills can’t be completely hardcoded. Imagine picking up a cup. Its position, weight, orientation and distance may be different from the last time. Mastery means adapting a learned skill to the current situation.

Vijay Sathish avatar
Vijay Sathish@Vslowsprinter·Oct 20

4/9 'Policy distillation' amortizes the costs for frequent tasks in the brain. When deliberation and refinement lead to mastery via repetition, the action policy becomes a repeatable skill requiring less conscious supervision. Practice makes execution cheaper.

Vijay Sathish avatar
Vijay Sathish@Vslowsprinter·Oct 20

3/9 Therefore, deliberation is effortful and computationally expensive. It makes economic sense to recruit this analytical machinery only when a novel situation, complexity or high stakes demand it.

Vijay Sathish avatar
Vijay Sathish@Vslowsprinter·Oct 20

2/9 Planning and deliberation require recruiting diverse experts, simulating trajectories and evaluating counterfactuals. Cortical, memory and basal ganglia brain regions interact throughout. Longer horizon simulations are even more expensive.

Vijay Sathish avatar
Vijay Sathish@Vslowsprinter·Oct 20

1/9 The brain evolved hierarchical planning and control. Why? Because necessity is the mother of invention. And because the brain is energetically and compute constrained necessitating extreme efficiency.

Vijay Sathish avatar
Vijay Sathish@Vslowsprinter·Oct 20

Robotics stole hierarchical control from the brain. But a significant gap still exists. Read on to see what’s missing 🧵 👇🏽 (Follow me (@Vslowsprinter) for regular AI x bio insights; only images are AI gen; thoughts my own)

Post attachment