←
Max S. Bennett148 posts
Max S. Bennett avatar

Max S. Bennett

@maxsbennett

Neuroscience & AI research @ Columbia and the Astera Institute, author of "A Brief History of Intelligence"

◷ Joined September 2018▧ 3 Photos and videos
Max S. Bennett avatar
Max S. Bennett@maxsbennett·Nov 9

I'm v excited to join @AsteraInstitute as a part-time research scientist. @dileeplearning has been a friend for years; in fact, it was his PhD thesis years ago that first got me into neuroAI. Excited to work with him and the amazing team he has assembled. Stay tuned for 🧠+🤖

Doris Tsao avatar
⟳ Max S. Bennett retweeted
Doris Tsao@doristsao·Oct 18

The outpouring of interest in our paper has been … a bit insane. May I make four points? 1. We have not shown that “consciousness has a frame rate.” I can see why people draw this conclusion: we report that inferotemporal and prefrontal cortex alternate between conscious and suppressed states roughly every 60 ms during binocular rivalry. Consciousness depends on many brain areas, and our paper reports results from only a few select nodes. We still don’t know how conscious access works--the actual recipe for generating conscious experience. What we have shown is that deep in the brain there is an extremely strong signal, rich enough to reconstruct a high-dimensional face in detail, that is inaccessible to consciousness. As I argued in the original tweet, this makes the problem of conscious access a lot more experimentally tractable. But to claim “consciousness has a frame rate,” one would need to have actually solved the mechanism for conscious access and shown that this mechanism operates at a frame rate. While we currently don’t know conclusively one way or the other (see VanRullen & Koch, 2003 for a review of positive evidence), I personally am skeptical that consciousness has a frame rate. I believe the brain, within posterior parietal cortex, has continuous tracking mechanisms that actively represent the persistence of matter, including ourselves as physical bodies. The coupling between these persistent tracking processes and feature representations across the brain (like those in face patches) is, I think, the essential mechanism of conscious access. This coupling need not happen at the rate of inferotemporal alternations (and inferotemporal cortex itself may not even alternate outside of the artificial situation of binocular rivalry). As an analogy: imagine the area crucial for conscious access holding a steady drone note, while inferotemporal cortex alternates between two notes, one in harmony and one dissonant. Inferotemporal content might couple to the drone only at moments of harmony, yet experience would still be continuous, because the drone itself (the tracking process that I believe is at the core of conscious perception) never stops. And areas without conflicting content, like auditory cortex during visual rivalry, could stay in continuous harmony with the drone. In sum, there is no reason to conclude from our study that consciousness is discrete, with a "frame rate." 2. Our results resemble Jezek et al. 2011 (as @jjsakon notes). Jezek et al. trained rats in two boxes that were identical except for their lighting cues, so that hippocampal place cells formed two distinct maps, one per box. They then “teleported” the rat by abruptly switching the lights. For a few seconds afterwards, CA3 place-cell activity flickered between the two maps, synced to the theta cycle (~125 ms). @StanDehaene first pointed this paper out to me, and reading it blew me away. To my mind it is the strongest evidence for a bistable attractor in the brain, because the competing maps are stored within the hippocampus itself. I think the phenomenon we see may be very similar at the circuit level: a bistable attractor network within inferotemporal cortex that alternately converges to one of two patterns, as cells in face patches work together to build a coherent representation of the input. In our case, both images are present in the input, so the competition could arise within IT or be inherited from earlier areas such as V1--recording there is an important next step. (It also still puzzles me that we didn’t see strong coherence between face patches ML and AM.) More broadly, what’s different about our paper is the brain area (inferotemporal cortex, often modeled as a feedforward DNN) and the ability to relate the two competing patterns to consciousness, with one granted conscious access and the other not. 3. Please comment on bioRxiv! The Twitter attention has been cool, but I really hope colleagues will be generous enough to post detailed commentary on bioRxiv, where there are currently 0 comments. As @seemaychou points out, peer review doesn’t have to come from just three anonymous reviewers. We would be truly grateful for your critical feedback. 4. Stay tuned This work was done at Caltech, Berkeley, in collaboration with @EngelTatiana at Princeton, funded by NIH and HHMI. I hope to share our vision soon for how we can solve consciousness @AsteraInstitute

1/ We discovered something totally unexpected about the neural structure of consciousness that overturns decades of thinking on its biological basis and brings us significantly closer to the recipe for generating conscious awareness. New preprint: biorxiv.org/content/10.64898… (led by @HesseJanis, with @FrankLanfranchi, @_conor_mcgrory, @Yuelin_Shi, and @EngelTatiana)

Max S. Bennett avatar
Max S. Bennett@maxsbennett·Sep 3

Everyone keeps beating all the AGI benchmarks. Where is the cure for cancer, alzheimers, etc 😒

Albert Wenger 🌎🔥⌛ avatar
⟳ Max S. Bennett retweeted
Albert Wenger 🌎🔥⌛@albertwenger·Apr 10

A Brief History of Intelligence by @maxsbennett is utterly wonderful in its clarity. A must read for anyone in both natural and artificial intelligence bookshop.org/p/books/a-brief…

Max S. Bennett avatar
Max S. Bennett@maxsbennett·Feb 27

So excited that this is happening. To my knowledge, this will be the most well funded neuroscience-inspired AGI initiative out there.

Dileep George @dileeplearning Feb 25 Article In search of the mystery of the cortical column and human-like general intelligence Today, Astera Institute announced that I am leading its AGI research program with an ambitious research agenda. It’s backed by $1B+ in long-term financial support for building NeuroAI that learns,

Max S. Bennett avatar
Max S. Bennett@maxsbennett·Feb 27

We believe long-term, controllable memory will be essential for: -Healthcare agents -Enterprise copilots -Continual deployment in non-stationary environments Was a blast collab'ing with @SquareZollo and Rich Zemel on this

Max S. Bennett avatar
Max S. Bennett@maxsbennett·Feb 27

Interestingly, selectivity isn’t just happening at inference time. The model learns where to write information into memory given the learning instruction. Early layers encode the instruction. Later layers align memory toward the target and away from distractors.

Max S. Bennett avatar
Max S. Bennett@maxsbennett·Feb 27

Key result: Neural memory can be controlled in natural language, and it generalizes to new learning instructions it has never seen before. It learns: what to learn what to ignore And it’s more selective than ICL and RAG.

Max S. Bennett avatar
Max S. Bennett@maxsbennett·Feb 27

So we built Generalized Neural Memory (GNM). Each document is paired with a natural-language instruction like: “Learn new medical escalation behavior, but ignore dosing.” or “Adopt this formatting, but don’t learn facts.” The memory update is conditioned on the instruction.

Max S. Bennett avatar
Max S. Bennett@maxsbennett·Feb 27

If we want AI systems that are steerable lifelong learners, instead of perfectly curated their training data, it is often easier for users to merely express what to learn and what not to learn.

Max S. Bennett avatar
Max S. Bennett@maxsbennett·Feb 27

Real documents contain mixed signals — only some of which are relevant depending on the task. Specific Facts. Style & Tone. Examples. Private information.