←
Ismini Lourentzou450 posts
Ismini Lourentzou avatar

Ismini Lourentzou

@Ismini_L

Director, PLAN Lab @iSchoolUI · NeurIPS'26 & ICML'26 Organizing Committee · GenAI, VLMs, Embodied AI, robots 🤖 | PhD @illinoisCDS 🇬🇷 plan-lab.github.io

◷ Joined September 2013▧ 119 Photos and videos
Ismini Lourentzou avatar
Ismini Lourentzou@Ismini_L·May 24

I am honored and grateful to receive an 𝗡𝗦𝗙 𝗖𝗔𝗥𝗘𝗘𝗥 award “𝘚𝘩𝘢𝘱𝘪𝘯𝘨 𝘌𝘮𝘣𝘰𝘥𝘪𝘦𝘥 𝘐𝘯𝘵𝘦𝘭𝘭𝘪𝘨𝘦𝘯𝘤𝘦 𝘛𝘩𝘳𝘰𝘶𝘨𝘩 𝘓𝘢𝘯𝘨𝘶𝘢𝘨𝘦-𝘎𝘶𝘪𝘥𝘦𝘥 𝘐𝘯𝘵𝘳𝘰𝘴𝘱𝘦𝘤𝘵𝘪𝘰𝘯.” 🎉 This project will support our work in the 𝗣𝗟𝗔𝗡 𝗟𝗮𝗯 toward more reliable, self-improving embodied AI systems. I am deeply thankful to @NSF, my students, whose creativity and hard work inspire me, as well as my collaborators, mentors, and @iSchoolUI colleagues for their support. I am also grateful to my former academic home @VT_CS where this work began to take shape. Excited for the work ahead! Learn more about our lab 👉 plan-lab.github.io #NSFCAREER #NSFfunded #EmbodiedAI #MultimodalAI #ResponsibleAI #iSchoolUI #PLANLab

#iSchoolUI Assistant Professor Ismini Lourentzou has received a $600K @NSF CAREER award for her project, “Shaping Embodied Intelligence Through Language-Guided Introspection,” which aims to build safer and more reliable AI systems. 🤖 Congrats, Ismini! ▶️ bit.ly/42QoWjn

Post attachment
Ismini Lourentzou avatar
Ismini Lourentzou@Ismini_L·Nov 17

Thank you for sharing our work @LeoKharon and thank you to @berkeley_ai for hosting @Jensen_Yuan 🙌

NEW RESEARCH: Predicting tuch is actually more useful than merely reading it! This project involves @Ismini_L and others Called DexTacWAM, for A Visuo-Tactile World-Action Model for Dexterous Manipulation, it is a method from UC Berkeley, UIUC, and Northwestern. It puts fingertip touch inside a video world model rather than beside it. Each fingertip's grayscale tactile map is painted as pseudo-RGB and pushed through a frozen pretrained VAE, the same encoder used for the camera streams and shared across all ten fingers. A compressor with finger-identity embeddings, hand-pose conditioning, self-attention pooling and temporal refinement folds five fingertip streams per hand into a single latent, cutting DiT tokens from 3,168 to 864. Those tactile latents are appended as extra views to a 2B-parameter pretrained video diffusion transformer, which is then trained with a modality-split flow-matching objective to denoise both the visual and the tactile future. An action expert cross-attends to the world model's latents with per-modality K/V RMS normalization. Training runs in three stages: 1. adapt the tactile encoder on a 488-episode, roughly four-hour bimanual corpus called Diverse-488 with the VAE frozen 2. finetune the world model on about 100 demonstrations per task 3. then train a fresh action expert on the same demonstrations. The platform is a 22-DoF bimanual rig with Sharpa hands and vision-based fingertip tactile sensors, head and wrist cameras. I find it super interesting that feeding the model touch does not bring much value, while making it predict touch actually enhances its performance.

0:35
Haoran Yuan✈️Corl avatar
⟳ Ismini Lourentzou retweeted
Haoran Yuan✈️Corl@Jensen_Yuan·Nov 10

I will give a talk tonight about VTAM and DexTamWAM with focus at Tactile Foundation Model. Welcome come to join!

Post attachment

Can a world-action model learn to predict touch without any tactile pretraining? We introduce VTAM, an efficient visuo-tactile world-action model. With ~100 demos per task and no tactile or action pretraining, it is comparable to tactile models pretrained on 30K+ hours of visuo-tactile robot data, and is 6× faster at inference. 🧵 📄Paper: arxiv.org/abs/2603.23481 🌐Web: plan-lab.github.io/projects/… Code and data are open source! 💻 Code: github.com/PLAN-Lab/VTAM 🤗 Data (xArm + GelSight, 3 tasks): huggingface.co/datasets/Jens…

1:55
SB avatar
⟳ Ismini Lourentzou retweeted
SB@SarvoCodes·Nov 7

VTAM uses an extra training target to encourage the robot policy to use touch. It summarizes image motion and expansion. Contact cues, not force readings in newtons. @Jensen_Yuan and team's Nov 5 revision. Synthetic example below. arxiv.org/abs/2603.23481v2

0:12
Ismini Lourentzou avatar
Ismini Lourentzou@Ismini_L·Nov 7

Proud of @Jensen_Yuan for leading this work! 👏 🤖VTAM is our visuo-tactile world-action model for contact-rich manipulation, trained with ~100 demos per task and no tactile pretraining. Code and data are open-sourced👇

Can a world-action model learn to predict touch without any tactile pretraining? We introduce VTAM, an efficient visuo-tactile world-action model. With ~100 demos per task and no tactile or action pretraining, it is comparable to tactile models pretrained on 30K+ hours of visuo-tactile robot data, and is 6× faster at inference. 🧵 📄Paper: arxiv.org/abs/2603.23481 🌐Web: plan-lab.github.io/projects/… Code and data are open source! 💻 Code: github.com/PLAN-Lab/VTAM 🤗 Data (xArm + GelSight, 3 tasks): huggingface.co/datasets/Jens…

1:55
Haoran Yuan✈️Corl avatar
⟳ Ismini Lourentzou retweeted
Haoran Yuan✈️Corl@Jensen_Yuan·Nov 6

Can a world-action model learn to predict touch without any tactile pretraining? We introduce VTAM, an efficient visuo-tactile world-action model. With ~100 demos per task and no tactile or action pretraining, it is comparable to tactile models pretrained on 30K+ hours of visuo-tactile robot data, and is 6× faster at inference. 🧵 📄Paper: arxiv.org/abs/2603.23481 🌐Web: plan-lab.github.io/projects/… Code and data are open source! 💻 Code: github.com/PLAN-Lab/VTAM 🤗 Data (xArm + GelSight, 3 tasks): huggingface.co/datasets/Jens…

1:55
Haoran Yuan✈️Corl avatar
⟳ Ismini Lourentzou retweeted
Haoran Yuan✈️Corl@Jensen_Yuan·Nov 4

Astra weight edit feels like another form of Richard Sutton-style continual learning: different parameters should adapt at different rates during deployment. Here, most parameters effectively have a step size of 0, while only a tiny subset is updated. Sparse weight editing may be a practical path toward continual adaptation without full-model fine-tuning in real-world deployment.

We test Astra weight edit on three robot domains: VLA, WAM and humanoid control. On one task it edited only 𝟒 𝐩𝐚𝐫𝐚𝐦𝐬 𝐨𝐮𝐭 𝐨𝐟 𝟑.𝟔𝐁 for π0.5, and success went 𝟐𝟖/𝟗𝟐 → 𝟖𝟐/𝟗𝟐 on held-out unseen episodes. How Astra got there actually interests me more 👇

0:32
Ismini Lourentzou avatar
Ismini Lourentzou@Ismini_L·Oct 29

I will be sharing an overview of our research on 𝐆𝐞𝐧𝐞𝐫𝐚𝐭𝐢𝐯𝐞 𝐖𝐨𝐫𝐥𝐝 𝐌𝐨𝐝𝐞𝐥𝐬 at the @iSchoolUI Research Showcase tomorrow, Oct. 30. 📍Illini Union, Rooms A & B ⌚2:15–3:15 p.m. CT Come learn about world models and robotics!

Join us for the #iSchoolUI Research Showcase on Oct. 30, from 11:30 a.m.–4:45 p.m. at the Illini Union! Explore human-centered, interdisciplinary research through keynote talks, short presentations, and posters. All are welcome! Full schedule ▶️ bit.ly/47zLdnV

Post attachment
NeurIPS Conference avatar
⟳ Ismini Lourentzou retweeted
NeurIPS Conference@NeurIPSConf·Oct 28

Both Sydney & Atlanta waitlists were activated on Oct. 27 at 10 am PDT. More registrations will be released on Oct. 30 10am PDT. Authors with unused reserved registration spots are reminded to register urgently.

Marco Pavone avatar
⟳ Ismini Lourentzou retweeted
Marco Pavone@drmapavone·Oct 26

How reliably can today’s robot policies complete tasks when the world changes mid-execution? In real-world robotics, things rarely go exactly as planned. Objects move, unexpected obstacles appear, and observations change as robots interact with their surroundings. Understanding how well robot policies handle these changes is essential for deploying Physical AI in the real world. In our new paper, LIBERO-MAX: Do Robot Policies Adapt When the World Changes?, we introduce a benchmark designed to study this question. We evaluate 14 state-of-the-art robot policies across 8,000 cases, introducing controlled changes after the robot has already begun acting. The results suggest that adapting to a changing world remains a key challenge for today’s robot policies. Task success drops by 11–26 percentage points, particularly when geometry or observations change, and simply updating policies more frequently does not close the gap. LIBERO-MAX provides a systematic way to measure these challenges, helping identify where current policies fall short and guiding the development of more adaptive and robust Physical AI systems. 📄 Paper: arxiv.org/abs/2609.36518 🌐 Benchmark and project website: liberomax.github.io/ 💻 Code: github.com/liberomax/LIBERO-… Work led by @YunbeiZhang, in collaboration with Zijian Jin, Yuanzhe Liu, Janet Wang, Xilun Zhang, Yuyou Zhang, Zhenyu Zhang, Daoan Zhang, Shuaicheng Niu, Gen Li, Jianfei Yang, Jihun Hamm, Ismini Lourentzou, Weirui Ye, Bo Liu, and Peter Stone. #Robotics #PhysicalAI #RobotLearning #RobotFoundationModels #AutonomousSystems

Post attachment
Yufei Wang avatar
⟳ Ismini Lourentzou retweeted
Yufei Wang@YufeiWang25·Oct 24

Life update: I recently graduated from CMU RI, and last year I went through the academic faculty job market in robot learning. I was fortunate to receive a lot of help and support from many people throughout the process. Compared to industry job applications, there are relatively few resources available online about the faculty application process. So I decided to open-source all of my application materials from last year here: docs.google.com/document/d/1… I hope these materials can be helpful to others who are considering pursuing an academic career!

Bryan Chan avatar
⟳ Ismini Lourentzou retweeted
Bryan Chan@cnm13ryan·Oct 22

New blogpost: Fable-5.1 can hide secret messages in benign images. We gave the image to another Fable in a fresh session and simply asked: "Is there a hidden message in this image?" It recovered the exact message without being told how it was hidden.

Post attachment