←
Thomas O'Connell79 posts
Thomas O'Connell avatar

Thomas O'Connell

@thomaspocon

cognitive neuroscientist @KanwisherLab, @mitcocosci

◷ Joined January 2017▧ 11 Photos and videos
Patrick Mineault avatar
⟳ Thomas O'Connell retweeted
Patrick Mineault@patrickmineault·2 Dec 2024

Excited to release what we’ve been working on at Amaranth Foundation, our latest whitepaper, NeuroAI for AI safety! A detailed, ambitious roadmap for how neuroscience research can help build safer AI systems while accelerating both virtual neuroscience and neurotech. 1/N

Post attachment
Ilia Sucholutsky avatar
⟳ Thomas O'Connell retweeted
Ilia Sucholutsky@sucholutsky·3 Nov 2023

🧵🎉 Our new preprint is up, and we’d love your feedback! We're "Getting Aligned on Representational Alignment" - the degree to which internal representations of different (biological & artificial) information processing systems agree. 🧠🤖🔬🔍 #CognitiveScience #Neuroscience #AI

Post attachment
Thomas O'Connell avatar
Thomas O'Connell@thomaspocon·13 Sep 2023

Finally, check out related work from my awesome co-author @tylerraye implicating the medial temporal lobe in supporting human 3D inferences. He suggests computations beyond standard DNNs are needed to model this process. Hmmm potential synergy? 🤔

excited to share my last experimental project from graduate school 😅🥹 "Medial temporal cortex supports compositional visual inferences" with my PhD advisors Anthony Wagner & Dan Yamins 1/🧠 biorxiv.org/content/10.1101/…

Thomas O'Connell avatar
Thomas O'Connell@thomaspocon·13 Sep 2023

tldr: DNNs trained with multi-view objectives make human-aligned 3D shape judgements! Lots more work to do here (biologically-plausible learning, generalization) to reach full human capabilities, but we take an important step closing the gap between human and model 3D inferences

Thomas O'Connell avatar
Thomas O'Connell@thomaspocon·13 Sep 2023

So, are we done? Not quite…none of these models generalize well to novel categories not included in their training set. Closing this generalization gap will be a focus of future research

Post attachment
Thomas O'Connell avatar
Thomas O'Connell@thomaspocon·13 Sep 2023

Remarkably, even a standard resnet50 CNN architecture showed a marked jump in alignment to humans when trained with a multi-view learning objective (Multi-View CNN)

Thomas O'Connell avatar
Thomas O'Connell@thomaspocon·13 Sep 2023

Models trained with a 3D multi-view objective (Multi-View Autoencoder & Multi-View CNN), in which two images depicting the same object from different viewpoints must be associated, were markedly more aligned to humans, approaching the performance of the 3D LFNs!

Post attachment
Thomas O'Connell avatar
Thomas O'Connell@thomaspocon·13 Sep 2023

We rule out training on rendered shapes (all control models), generative capabilities (Autoencoder), and viewpoint supervision (Autoencoder+Viewpoints)  as sufficient for learning human-aligned 3D shape representations (see plot below)

Thomas O'Connell avatar
Thomas O'Connell@thomaspocon·13 Sep 2023

Now for the fun part! What drives alignment between 3D LFNs and humans? To figure this out, we trained a series of control models, each incorporating different aspects of 3D LFNs into more standard architectures

Thomas O'Connell avatar
Thomas O'Connell@thomaspocon·13 Sep 2023

To ensure these results aren’t driven by the category structure of ShapeNet, we repeat the procedure using abstract procedurally-generated shapes (top) created with the ShapeGenerator plugin for Blender. Again, we the 3D LFN is most aligned to human 3D shape judgements (bottom)

Post attachment
Thomas O'Connell avatar
Thomas O'Connell@thomaspocon·13 Sep 2023

Next, we construct adversarial match-to-sample trials by minimizing the accuracy for 25 ImageNet CNNs and selecting trials across 5 difficulty conditions. Even for these adversarially-defined trials, alignment between human and 3D LFN 3D shape judgements holds

Post attachment