←
vislang.ai94 posts
vislang.ai avatar

vislang.ai

@vislang

Twitter account of the Vision, Language and Learning Lab, Computer Science @ Rice University.

◷ Joined June 2020▧ 14 Photos and videos
Moayed Haji Ali avatar
⟳ vislang.ai retweeted
Moayed Haji Ali@moayedhajiali·Mar 13

Not all pixels are equally hard, but DiTs still allocate compute uniformly across pixels, wasting efforts on easy regions. ELIT adds two lightweight cross-attention layers to focus compute where it matters, cutting FID by 53%. ELIT: snap-research.github.io/elit

Post attachment
Guilherme Favaron avatar
⟳ vislang.ai retweeted
Guilherme Favaron@guifav·Mar 15

Diffusion transformers waste compute by treating every pixel equally, regardless of content complexity. ELIT (Elastic Latent Interface Transformer) fixes this with a simple idea: insert a variable length set of latent tokens that learn where to spend computation. Two lightweight cross attention layers (Read/Write) route information between spatial tokens and latents. The model learns importance ordering during training by randomly dropping tail latents, so earlier tokens capture global structure while later ones handle fine details. Results on ImageNet 1K at 512px: 35.3% better FID, 39.6% better FDD scores, ~33% cheaper classifier free guidance. Works across DiT, UViT, HDiT, and MMDiT architectures with no changes to the training objective. By Moayed Haji Ali, @vislang (Rice University), @SergeyTulyakov, Aliaksandr Siarohin, Willi Menapace, Ivan Skorokhodov and team at @Snap. Accepted at CVPR 2026.

Post attachment
Zilin Xiao avatar
⟳ vislang.ai retweeted
Zilin Xiao@ZilinXiao2·Feb 6

🚀 Two papers accepted to #ICLR2026 on test-time scaling for vision-language systems (retrieval + reasoning)! 1) MetaEmbed (Oral Presentation): Meta Tokens + Matryoshka multi-vector training → flexible late interaction, choose #vectors at test time for accuracy↔efficiency. Paper: arxiv.org/abs/2509.18095 Work done at @AIatMeta with amazing collaborators: Qi Ma, @Mengting_Gu, Jason Chen, Xintao Chen, @vislang and @MohanVijaimohan! 2) ProxyThinker: training-free test-time guidance from small “slow-thinking” visual reasoners → self-verification / self-correction via distribution-level guidance. Paper: arxiv.org/abs/2505.24872 Work done with @JaywonK17250, @Siru_Ouyang, @jefehern, @yumeng0818 and @vislang! While I won't be able to travel to Brazil🇧🇷, please say Hi to the team :-) #MultimodalRetrieval #VisualReasoning #VisionLanguage #TestTimeCompute #Embeddings

AK avatar
⟳ vislang.ai retweeted
AK@_akhaliq·23 Sep 2025

MetaEmbed Scaling Multimodal Retrieval at Test-Time with Flexible Late Interaction

Post attachment
Aran Komatsuzaki avatar
⟳ vislang.ai retweeted
Aran Komatsuzaki@arankomatsuzaki·23 Sep 2025

Meta Superintelligence Labs presents MetaEmbed: Scalable multimodal retrieval • Flexible late interaction via Meta Tokens • Test-time scaling: trade off retrieval accuracy vs efficiency • SOTA on MMEB + ViDoRe, robust up to 32B models • Matryoshka training → coarse-to-fine multi-vector embeddings

Post attachment
AK avatar
⟳ vislang.ai retweeted
AK@_akhaliq·15 Dec 2021

CLIP-Lite: Information Efficient Visual Representation Learning from Textual Annotations abs: arxiv.org/abs/2112.07133 CLIP-Lite obtains a +15.4% mAP absolute gain in performance on Pascal VOC classification, and a +22.1% top-1 accuracy gain on ImageNet

Post attachment
Zilin Xiao avatar
⟳ vislang.ai retweeted
Zilin Xiao@ZilinXiao2·11 Jun 2025

Looking for a new (image) re-ranking paradigm? Check this out! LoCoRe (Long-Context Reranker) is trained with a long-context sequence model and token-level supervision to achieve **one-pass** re-ranking for all image candidates. Catch us at #CVPR Poster Session 2 #401 on Friday, 4pm-6pm!

Post attachment
vislang.ai avatar
vislang.ai@vislang·14 Jan 2025

Check our new work on cross-modal audio-video generation. Our work produces audio with the best alignment we have seen with respect to actions happening on video. Particularly useful in the era of astounding progress in generative video models.

Can pretrained diffusion models connect for cross-modal generation? 📢 Introducing AV-Link ♾ Bridging unimodal diffusion models in one framework to enable: 📽️ ➡️ 🔊 Video-to-Audio 🔊 ➡️ 📽️ Audio-to-Video 🌐: snap-research.github.io/AVLi… 📄: hf.co/papers/2412.15191 ⤵️ Results

0:05
Reginald DesRoches avatar
⟳ vislang.ai retweeted
Reginald DesRoches@RDesRoches·4 Oct 2024

Rice is shaping the future of AI! Our researchers are working on groundbreaking methods to eliminate the "weird" or distorted images that AI sometimes generates. This innovation could lead to more accurate and realistic visuals created by artificial intelligence. The future of AI-generated imagery is looking clearer and brighter thanks to our researchers! 🔍💡Read more about this fascinating research and its potential impact: bit.ly/4ezY37m #RiceU #FutureOfAI #Innovation

Post attachment
Zilin Xiao avatar
⟳ vislang.ai retweeted
Zilin Xiao@ZilinXiao2·28 Sep 2024

I am excited to share that two of our research works will be presented at ECCV 2024. #ECCV2024 They focus on augmenting language models with fine-grained visual recognition ability. AutoVER made successful attempts at generative visual recognition. It was accepted to the ECCV 2024 main conference and was invited to the ILR Workshop as an oral presentation. Collaboration w/ @pcascanteb @vislang #Microsoft Extractive Reranker was accepted to the ILR Workshop as a poster. We explored how the long-context sequence modeling ability of language models can benefit image retrieval, a fundamental computer vision problem.

Post attachmentPost attachmentPost attachment
vislang.ai avatar
vislang.ai@vislang·19 Jun 2024

Check out the work by @cathyrzhe who is representing our group at #CVPR2024 at the poster session tomorrow morning. Poster #334. #CVPR.

(1/4) Excited to share our latest work at #CVPR2024 @CVPR!🔥 Join us tomorrow, Thursday, June 20, from 10:30am to noon at Poster Session 3, # 334, to learn about "Improved Visual Grounding through Self-Consistent Explanations" with @pcascanteb, Ziyan, @alexandercberg, @vislang.

Post attachment