←
Max450 posts
Max avatar

Max

@bfpill

research @resolution_org previously @ Melbourne uni

◷ Joined December 2023▧ 55 Photos and videos
Max avatar
Max@bfpill·Nov 10

Norbert Wiener, a forefather of control theory and arguably the first alignment researcher, warned in 1960 that we had better be sure “the purpose put into the machine is the purpose which we really desire.” New post: patterning as control (in the Toy Model of Superposition).

0:43
Max avatar
Max@bfpill·4h

how are we feeling about catastrophic near futures today folks

Jesse Hoogland avatar
⟳ Max retweeted
Jesse Hoogland@jesse_hoogland·Nov 17

I sat down with @hilbertspaess for two hours on RSI, the METR report, AI extinction risk, and Anthropic. Jacob comes off great. He's an extremely lucid speaker, and it was a delight to get to interview him. I was edited by my brother.

The Longest Interview with Jacob Coxon. Featuring @hilbertspaess & @jesse_hoogland Link below:

2:51
Max avatar
Max@bfpill·Nov 10

Getting the sorts of controllability guarantees we want for alignment will likely involve understanding the interaction between singularities in the parameter function map and changes to our controls (data). This question is both theoretically and empirically rich!

Max avatar
Max@bfpill·Nov 10

For harder targets one step isn’t enough, and so we repeatedly compute and apply perturbations to the training distribution. Hence patterning traces a path through data and parameter space.

0:45
Max avatar
Max@bfpill·Nov 10

The susceptibilities of the 4-gon tell us that upweighting inputs where only f0 is active will push the new feature away from f0. By combining pushes like this, we can tilt the landscape towards any of the four gaps.

0:31
Max avatar
Max@bfpill·Nov 10

So how should we break it? In physics, susceptibilities tell us how a system responds to a small push (like a magnet in a weak field). Here, our system is a neural network and our push is a small change in its training distribution.

0:45
Max avatar
Max@bfpill·Nov 10

Learning is a walk down a chain of states à la Waddington’s landscape. Starting at the 4-gon (the square), there are four slots for the fifth feature to grow into, each equally likely under the training data. To choose which gap is taken, a symmetry of the data needs to be broken

0:38
Max avatar
Max@bfpill·Nov 10

In our new post, we demonstrate patterning works using the Toy Model of Superposition (TMS), a beautiful model where the developmental process is made ‘plane’. The TMS tries to represent more features than it has dimensions—here, it has two dimensions to represent five features.

0:31
Max avatar
Max@bfpill·Nov 10

Alignment is about controlling generalization. Patterning is a control method designed to control generalization. Control, control, control! Control, for us, means being able to choose properties of the algorithms found by the learning process. resolution.org/post/patterni…

Jesse Hoogland avatar
⟳ Max retweeted
Jesse Hoogland@jesse_hoogland·Oct 6

1/ We have autoformalized the resolution of singularities in Lean. Hironaka's 1964 theorem is one of the great mathematical results of the 20th century. It says that any singular variety is the "shadow" of a smooth one living in higher dimensions. So why did we do this? 🧵

Geoffrey Irving avatar
⟳ Max retweeted
Geoffrey Irving@geoffreyirving·Oct 3

There are numerous scientific uncertainties about AI that will not resolve until it is too late to act on the answers. The danger from AI is very high, and we should stop frontier AI scaling now, instead of waiting for these questions to resolve. I wrote about this for TIME. 🧵

Post attachment
Max avatar
Max@bfpill·Sep 29

drawing a line in the muck at the ninajirachi portola set separating the before-times and the after-times