←
Charlie S. Burlingham133 posts
Charlie S. Burlingham avatar

Charlie S. Burlingham

@csburlingham

Vision Science & AI @ Meta Reality Labs

◷ Joined April 2021▧ 5 Photos and videos
Charlie S. Burlingham avatar
Charlie S. Burlingham@csburlingham·4 May 2025

🎉 New paper out! We show training improves motion categorization but doesn't reduce (or even worsens) misperceptions—explained via model combining efficient coding + implicit categorization + increased encoding precision journals.plos.org/ploscompbi…

Eiko Fried avatar
⟳ Charlie S. Burlingham retweeted
Eiko Fried@EikoFried·18 Jul 2024

So in 2007, physicists wrote a paper that made the headlines: according to their calculations, human coin flips aren’t 50/50 - more like 51/49. Why is that, and did students in Amsterdam really flip 350,000 coins to find out? 🧵

Charlie S. Burlingham avatar
Charlie S. Burlingham@csburlingham·2 Jul 2024

We've open-sourced the egocentric eye tracking dataset from our recent papers — via @Facebook @RealityLabs GitHub. ~26 hrs of user gaze and head movement data across 9 everyday tasks: driving, shopping, LEGO, Pokemon GO, + more. github.com/facebookresearch/… h/t @tsmurdison

Airi Yoshimoto avatar
⟳ Charlie S. Burlingham retweeted
Airi Yoshimoto@AiriSyoshimoto·20 Jun 2024

I’m very excited to share that my graduate work is now online in @ScienceMagazine today! With generous help from my mentor @yuji_ikegaya and my amazing teammates, we investigated a top-down pathway for volitional heart rate regulation! science.org/doi/10.1126/scie…

Michael J. Proulx avatar
⟳ Charlie S. Burlingham retweeted
Michael J. Proulx@MichaelProulx·12 Jun 2024

All-day AR would benefit from AI models that understand a person's context & eye tracking could be key for task recognition. Yet past work - including our own research.facebook.com/public… - hasn't found much added value from gaze in addition to computer vision & egocentric video 2/

Leah Banellis avatar
⟳ Charlie S. Burlingham retweeted
Leah Banellis@LeahBanellis·10 Jun 2024

Got Butterflies in your Stomach?😵‍💫I am super excited to share the first major study of my postdoc @visceral_mind! We report a multidimensional mental health signature of stomach-brain coupling in the largest sample to date 🧵👇biorxiv.org/content/10.1101/…

Charlie S. Burlingham avatar
Charlie S. Burlingham@csburlingham·12 Jun 2024

Once the input state space is well-aligned with human action & vision, and appropriate models that can represent long-term dependencies are used, we believe that multiple problems in contextual AI may be solved convergently by a single (gaze-based) visual foundation model. 6/

Charlie S. Burlingham avatar
Charlie S. Burlingham@csburlingham·11 Jun 2024

But object-part-based image segmentation is just starting to gain traction, and universal segmentation (segmenting and labelling all image pixels) is still a challenge. So this is one major bottleneck for aligning the model input state space with that of human active vision. 5/

Charlie S. Burlingham avatar
Charlie S. Burlingham@csburlingham·11 Jun 2024

An important insight from years of work in active vision (Hayhoe, Land, Tatler, Yarbus) is that gaze fixations are highly specific. E.g., to fill a kettle when making tea we must look at the faucet handles when turning them, the inside of the kettle so it doesn't overflow, etc 4/

Charlie S. Burlingham avatar
Charlie S. Burlingham@csburlingham·11 Jun 2024

Like language, visual gaze across objects in our environment is predictable and compositional, enabling rich non-verbal communication and providing a window into our thoughts & goals. We argue that properly formatting this "gaze language" for contextual AI models is crucial. 3/

Charlie S. Burlingham avatar
Charlie S. Burlingham@csburlingham·11 Jun 2024

Therefore, scanpaths in real environments are non-Markovian and contain stereotyped, functional “motifs” with context-dependent timescales. These scanning motifs allow for gathering visual information and ensuring safe and effective action. 2/

Charlie S. Burlingham avatar
Charlie S. Burlingham@csburlingham·11 Jun 2024

Scanpaths are often assumed to obey a Markovian assumption. We analyzed scanpaths during nine everyday tasks and quantified long temporal dependencies using mutual information, with an average timescale of four fixations (2 seconds) into past (w/ large variation across tasks) 1/