←
Ke Li 🍁315 posts
Ke Li 🍁 avatar

Ke Li 🍁

@KL_Div

Associate Professor and Canada CIFAR AI Chair @SFU @AmiiThinks. Ph.D. from @Berkeley_EECS and Bachelor's from @UofTCompSci. Formerly @GoogleAI and @the_IAS.

◷ Joined June 2019▧ 124 Photos and videos
Ke Li 🍁 avatar
Ke Li 🍁@KL_Div·Sep 5

Diffusion/flow matching is a go-to generative AI paradigm, but it's slow, sample inefficient and complicated. Do we really need all this complexity? We design a small and simple one-step model, ROMS-IMLE, which attains an FID of 2.56 on ImageNet. Joint w/ @researchirag 1/6

0:20
Ke Li 🍁 avatar
Ke Li 🍁@KL_Div·Sep 27

Conditional IMLE (cIMLE) handles multimodal distributions by giving the model the opportunity to generate samples from multiple modes and avoiding penalizing samples from the modes that are different from the mode corresponding to the current observation. 7/8

1:09
Ke Li 🍁 avatar
Ke Li 🍁@KL_Div·Sep 27

IMLE-VLA reacts faster, exhibits less stop-and-go and produces less jerk compared to π0.5. 6/8

0:37
Ke Li 🍁 avatar
Ke Li 🍁@KL_Div·Sep 27

Moreover, flow matching leads to jerky motion because the flow map is invertible. Hence, it cannot collapse the noise distribution onto a manifold, because doing so would require an non-invertible transformation. cIMLE gets around this and can learn the precise manifold. 5/8

0:50
Ke Li 🍁 avatar
Ke Li 🍁@KL_Div·Sep 27

To predict the modes rather than the mean, π0.5 uses flow matching. Since it needs 10 Euler steps, each of which involves a forward pass through the action head, it is slow to react and produces stop-and-go motion. Our cIMLE action head takes one step and is 3.67x faster. 4/8

0:31
Ke Li 🍁 avatar
Ke Li 🍁@KL_Div·Sep 27

Simply regressing to the demonstrations doesn't work, because the optimal predicted action under least squares is the average across different modes. 3/8

0:59
Ke Li 🍁 avatar
Ke Li 🍁@KL_Div·Sep 27

VLAs turn two camera views and a natural language instruction into a sequence of actions to be executed on a robot. Because there are multiple valid ways to accomplish a task, the action distribution is multimodal. Handling this is a challenge. 2/8

0:46
Ke Li 🍁 avatar
Ke Li 🍁@KL_Div·Sep 27

VLA policies are slow and jerky. Introducing IMLE-VLA, which swaps out flow matching for conditional IMLE. IMLE-VLA generates action chunks at 55 Hz (3.67x faster than π0.5), produces >2.2x less jerk and finishes tasks >3.9x faster. Details at kianhk6.github.io/IMLE-VLA/ If you are at #IROS2026, don't miss @kianhk3's talk on Monday at 9am in Room 329. Joint work w/ @kianhk3, Qinhe Peng, @hekoshii, @mehranag, @jianing_qian, @tristanengst, @Moazeni_Alireza and @dineshjayaraman Thread 👇 1/8

0:37
Alireza Moazeni avatar
⟳ Ke Li 🍁 retweeted
Alireza Moazeni@Moazeni_Alireza·Sep 10

If you're at #ECCV2026, drop by our poster at ExHall #187. I'd love to walk you through Intrinsic-PAPR!

Editing appearance properties (e.g., colour, shading, albedo, etc.) of 3DGS scenes can yield unexpected results - after editing, the colour may be very off. We found it was caused by misattribution of ray colour to primitives, a side effect of volume rendering. PAPR gets around this by attending to all nearby points, which encourages points to move to the surface. Joint work w/ @Moazeni_Alireza, @PengShichong, @yszhang170 and @researchirag If you are attending #ECCV2026, stop by poster 187 this morning to learn more! Project Website: amoazeni75.github.io/Intrins… Paper: arxiv.org/abs/2407.00500 Code: github.com/amoazeni75/Intrin… Video: youtube.com/watch?v=l1wdWTg7…

0:56
Ke Li 🍁 avatar
Ke Li 🍁@KL_Div·Sep 10

Editing appearance properties (e.g., colour, shading, albedo, etc.) of 3DGS scenes can yield unexpected results - after editing, the colour may be very off. We found it was caused by misattribution of ray colour to primitives, a side effect of volume rendering. PAPR gets around this by attending to all nearby points, which encourages points to move to the surface. Joint work w/ @Moazeni_Alireza, @PengShichong, @yszhang170 and @researchirag If you are attending #ECCV2026, stop by poster 187 this morning to learn more! Project Website: amoazeni75.github.io/Intrins… Paper: arxiv.org/abs/2407.00500 Code: github.com/amoazeni75/Intrin… Video: youtube.com/watch?v=l1wdWTg7…

0:56
Yanshu Zhang avatar
⟳ Ke Li 🍁 retweeted
Yanshu Zhang@yszhang170·Sep 9

Excited to share another work at ECCV 2026! I’ll be presenting PointGT at #ECCV2026 tomorrow — on Thursday, Sep 10 at 10:30 AM – 12:30 AM poster # 206! Come by if you’re interested in texture or geometry editing in 3D. Would love to chat and hear your thoughts!

Editing texture in 3DGS results in artifacts that become especially pronounced when the geometry changes. Introducing Point-GT, which learns a UV map that adapts automatically to deformations with PAPR. Joint w/ @yszhang170, @hekoshii and @_pratul_. Project Website: zvict.github.io/pointgt/ Paper: arxiv.org/abs/2609.03341 Code: github.com/zvict/pointgt Video: youtube.com/watch?v=aHKiGW_o… If you are at #ECCV2026, come check out poster #206 this Thursday morning at 10:30am!

0:34
Ke Li 🍁 avatar
Ke Li 🍁@KL_Div·Sep 9

Editing texture in 3DGS results in artifacts that become especially pronounced when the geometry changes. Introducing Point-GT, which learns a UV map that adapts automatically to deformations with PAPR. Joint w/ @yszhang170, @hekoshii and @_pratul_. Project Website: zvict.github.io/pointgt/ Paper: arxiv.org/abs/2609.03341 Code: github.com/zvict/pointgt Video: youtube.com/watch?v=aHKiGW_o… If you are at #ECCV2026, come check out poster #206 this Thursday morning at 10:30am!

0:34
Yanshu Zhang avatar
⟳ Ke Li 🍁 retweeted
Yanshu Zhang@yszhang170·Sep 8

Excited to share our ECCV 2026 work, P-CORE! I’ll be presenting this work at #ECCV2026 — come chat with me on Thu, Sep 10 at 10:30 AM – 12:30 AM poster # 207, if you’re interested in neural rendering, 3D reconstruction, or 3D editing!

While 3D scenes can be rendered from new viewpoints with high fidelity, the rendering quality doesn't survive freeform deformations of the learned 3D representations. Our #ECCV2026 paper, P-CORE, fixes this. Joint work with @yszhang170 @PengShichong @mehranag @Moazeni_Alireza Details at zvict.github.io/p-core/ 1/5

0:08