←
Genta Winata1,261 posts
Genta Winata avatar

Genta Winata

@gentaiscool

AI Researcher @CapitalOne AIF. Ex @TechAtBloomberg @BigScienceW @SFResearch @hkust. Working on multilingual and LLM #NLProc.

◷ Joined December 2011▧ 98 Photos and videos
M. Rifqi Farhansyah avatar
⟳ Genta Winata retweeted
M. Rifqi Farhansyah@rifqifrhnsyah·Aug 21

🥳 Happy to share that PingPong has been accepted to EMNLP 2026 - Main Conference! Special thanks to all of my collaborators: @hanifmz07, @faridlazuarda, Maryam**, @Shmuhammadd, @nedjmaou, @sathready, @ponguru, @KodaliPrashant, Sunayana**, @gentaiscool, @AyuP_AI, @AlhamFikri

Post attachment

⚠️New Paper Alert⚠️ Introducing PingPong: A Natural Benchmark for Multi-Turn Code-Switching Dialogues Ever seen a 4-player ping-pong game with chaotic turn-taking? That’s a perfect analogy for multi-party polyglot in code-switching conversations. We introduce PingPong, a benchmark for natural multi-party code-switching dialogues covering five language-combination variations, some of which are trilingual and one that involves a different writing script.🏓🌍 📄Paper: arxiv.org/pdf/2601.17277 🤗Datasets: huggingface.co/code-switchin… Huge thanks to all of my collaborators: @hanifmz07, @faridlazuarda, @Shmuhammadd, Maryam**, @nedjmaou, @gentaiscool, @AyuP_AI, @AlhamFikri. Check this thread out! 🧵 #PingPongDataset #CodeSwitching #Multilinguality #Polyglot #NLP #LLM #Benchmark (1/n)

Post attachment
Mohit Bansal avatar
⟳ Genta Winata retweeted
Mohit Bansal@mohitban47·Jun 30

-- Routing with Generated Data: Annotation-Free LLM Skill Estimation and Expert Selection (Oral) @niu_tianyi , @cyjustinchen , @gentaiscool , Shi-Xiong Zhang, Supriyo Chakraborty, Sambit Sahu, @zhan1624 , @EliasEskin arxiv.org/abs/2601.09692 #ACL2026 Harbor D: Session 17: Oral Session G: Tues. July 7, 11:00-12:30 PT x.com/EliasEskin/status/2011…

📢 Introducing Routing with Generated Data (RGD), a new setting for annotation-free LLM routing. We study how routers can be trained without any ground-truth labels. We also introduce CASCAL, a novel label-free LLM router that identifies niche skills using consensus-voting and hierarchical clustering. ➡️ Most LLM routers assume access to labeled, in-domain data to estimate model skills (query-answer routers). However, user distributions are unknown and labels are expensive or unavailable, highlighting the need for routers that work without labels. ➡️ We introduce Routing with Generated Data (RGD): routers are trained only on Q&A data generated from task descriptions, without human annotation. We experiment with various LLM generators of different strengths (Gemini-2.5-Flash, Qwen-3-32B, Exaone-3.5-7.8B). ➡️ CASCAL outperforms other query-answer and query-only routers across diverse datasets (MMLU-Pro, SuperGPQA, MedMCQA, BigBench Extra Hard), and is more robust to weaker generators.

Post attachment
Elias Stengel-Eskin avatar
⟳ Genta Winata retweeted
Elias Stengel-Eskin@EliasEskin·Jul 2

2/ #ACL2026 Tues. July 7, 11:00-12:30 Harbor D: Session 17: Oral Session G Routing with Generated Data: Annotation-Free LLM Skill Estimation and Expert Selection @niu_tianyi, @cyjustinchen, @gentaiscool, Shi-Xiong Zhang, Supriyo Chakraborty, Sambit Sahu, @zhan1624, @mohitban47 arxiv.org/abs/2601.09692 x.com/EliasEskin/status/2011…

📢 Introducing Routing with Generated Data (RGD), a new setting for annotation-free LLM routing. We study how routers can be trained without any ground-truth labels. We also introduce CASCAL, a novel label-free LLM router that identifies niche skills using consensus-voting and hierarchical clustering. ➡️ Most LLM routers assume access to labeled, in-domain data to estimate model skills (query-answer routers). However, user distributions are unknown and labels are expensive or unavailable, highlighting the need for routers that work without labels. ➡️ We introduce Routing with Generated Data (RGD): routers are trained only on Q&A data generated from task descriptions, without human annotation. We experiment with various LLM generators of different strengths (Gemini-2.5-Flash, Qwen-3-32B, Exaone-3.5-7.8B). ➡️ CASCAL outperforms other query-answer and query-only routers across diverse datasets (MMLU-Pro, SuperGPQA, MedMCQA, BigBench Extra Hard), and is more robust to weaker generators.

Post attachment
Elias Stengel-Eskin avatar
⟳ Genta Winata retweeted
Elias Stengel-Eskin@EliasEskin·Apr 9

🚨 Excited to announce that RGD has been accepted to #ACL2026 Main! Routing with Generated Data (RGD) is a new LLM routing paradigm where routers estimate skills of models using generated data, without ground-truth labels. We further introduce CASCAL, a new router for RGD that discovers niche skills via consensus voting + hierarchical clustering, with no ground truth needed. 🧵👇

📢 Introducing Routing with Generated Data (RGD), a new setting for annotation-free LLM routing. We study how routers can be trained without any ground-truth labels. We also introduce CASCAL, a novel label-free LLM router that identifies niche skills using consensus-voting and hierarchical clustering. ➡️ Most LLM routers assume access to labeled, in-domain data to estimate model skills (query-answer routers). However, user distributions are unknown and labels are expensive or unavailable, highlighting the need for routers that work without labels. ➡️ We introduce Routing with Generated Data (RGD): routers are trained only on Q&A data generated from task descriptions, without human annotation. We experiment with various LLM generators of different strengths (Gemini-2.5-Flash, Qwen-3-32B, Exaone-3.5-7.8B). ➡️ CASCAL outperforms other query-answer and query-only routers across diverse datasets (MMLU-Pro, SuperGPQA, MedMCQA, BigBench Extra Hard), and is more robust to weaker generators.

Post attachment
Alham Fikri Aji avatar
⟳ Genta Winata retweeted
Alham Fikri Aji@AlhamFikri·Mar 15

VLMs can easily get distracted by unrelated cultural cues. Happy to present our work on this soon at #CVPR2026🥳 Working on multilingual VLMs? Consider using our benchmark: 📜arxiv.org/pdf/2511.17004 🤗huggingface.co/datasets/patr… Amazing work by @patrickamadeus_ and colleagues!

Post attachment

Excited to share that we have committed our paper “Vision-Language Models are Confused Tourists” to #CVPR2026 (Findings)! 🇺🇸🏔 Arxiv: arxiv.org/abs/2511.17004 We question whether current SOTA VLMs remain robust in simple cultural grounding QA when distracting contextual objects are present For example, if you eat chicken schnitzel with Mt. Fuji in the background, will the model fail to recognize it as Japanese katsu? ConfusedTourists introduces: 👉 5k+ evaluation samples across 3 cultural item categories, comprising 243 unique cultural items from 57 countries and 11 sub-regions 🌍 👉 Evaluation of 14 VLMs across 12 data features 🤖 👉 Findings showing that simple concept mixing can cause up to a -40% drop in perform 📉 Special thanks to my co-authors @IkhlasulHanif0 , @emthehunt, @gentaiscool, @FajriKoto, and my advisor @AlhamFikri for the valuable contributions along the way! #multimodal #vlm #multicultural #robustness #evaluation #NLProc #ComputerVision

Post attachmentPost attachmentPost attachment
Genta Winata avatar
Genta Winata@gentaiscool·Feb 2

Happy to have my first Nature paper. Thank you @CAIS for the collaboration nature.com/articles/s41586-0…

Last week, Humanity’s Last Exam was published in @Nature. In just over a year, model scores on HLE have risen from under 5% to nearly 40%. Thank you to @scale_AI and the 1000+ HLE co-authors for helping policymakers and the public track these rapid advances in AI capabilities.

Post attachmentPost attachment
Center for AI Safety avatar
⟳ Genta Winata retweeted
Center for AI Safety@CAIS·Feb 2

Last week, Humanity’s Last Exam was published in @Nature. In just over a year, model scores on HLE have risen from under 5% to nearly 40%. Thank you to @scale_AI and the 1000+ HLE co-authors for helping policymakers and the public track these rapid advances in AI capabilities.

Post attachmentPost attachment
Elias Stengel-Eskin avatar
⟳ Genta Winata retweeted
Elias Stengel-Eskin@EliasEskin·Jan 15

📢 Introducing Routing with Generated Data (RGD), a new setting for annotation-free LLM routing. We study how routers can be trained without any ground-truth labels. We also introduce CASCAL, a novel label-free LLM router that identifies niche skills using consensus-voting and hierarchical clustering. ➡️ Most LLM routers assume access to labeled, in-domain data to estimate model skills (query-answer routers). However, user distributions are unknown and labels are expensive or unavailable, highlighting the need for routers that work without labels. ➡️ We introduce Routing with Generated Data (RGD): routers are trained only on Q&A data generated from task descriptions, without human annotation. We experiment with various LLM generators of different strengths (Gemini-2.5-Flash, Qwen-3-32B, Exaone-3.5-7.8B). ➡️ CASCAL outperforms other query-answer and query-only routers across diverse datasets (MMLU-Pro, SuperGPQA, MedMCQA, BigBench Extra Hard), and is more robust to weaker generators.

Post attachment
Genta Winata avatar
Genta Winata@gentaiscool·24 Dec 2025

💡Have you ever wondered whether vision–language models can be easily tricked by adding landmarks or flags to an image? In the spirit of the holidays🎄, we show that VLMs can indeed be easily confused like "Confused Tourists" ✈️: their performance drops significantly when such image perturbations are applied. 🔎 Check out "VLMs are Confused Tourists" ✈️ here arxiv.org/pdf/2511.17004 #vision #nlproc #robustness

Craving holiday-themed paper? Say less🎄 Turns out, Vision Language Models are Confused Tourists ✈️😵‍💫 We show that adversarially induced cultural scenes significantly impair VLM cultural comprehension and trigger potential bias #NLProc #multimodal #robustness /thread 🧵(1/8)

Post attachment
patrick avatar
⟳ Genta Winata retweeted
patrick@patrickamadeus_·23 Dec 2025

Craving holiday-themed paper? Say less🎄 Turns out, Vision Language Models are Confused Tourists ✈️😵‍💫 We show that adversarially induced cultural scenes significantly impair VLM cultural comprehension and trigger potential bias #NLProc #multimodal #robustness /thread 🧵(1/8)

Post attachment
SEACrowd avatar
⟳ Genta Winata retweeted
SEACrowd@seacrowd_ai·16 Dec 2025

One last call for interested applicants! Applications for the SEACrowd Apprentice Program 2026 will end on December 17, 23:59 (UTC-12). See the post below for more details.

🌏 Applications are now open for the SEACrowd Apprentice Program 2026! Join a 3–4 month guided research journey where you’ll collaborate with mentors. 🗓️ Apply between Nov 17 – Dec 17, 2025 (UTC-12) 📅 Program runs Feb – Jun 2026 seacrowd.org/apprenticeship

SEACrowd avatar
⟳ Genta Winata retweeted
SEACrowd@seacrowd_ai·6 Dec 2025

We’ve just shared a full retrospective of the first batch our Apprentice Program from research wins to real lessons learned along the way. It’s inspiring to see how our SEA mentees transformed ideas into papers and datasets through mentorship.🚀🚀 seacrowd.org/posts/apprentic…

Stefan Horoi avatar
⟳ Genta Winata retweeted
Stefan Horoi@stefanhoroi·1 Dec 2025

Excited to be at #NeurIPS 2025 in San Diego this week! 🌴🤖 I’m presenting our UniReps workshop paper on Dec 6: Leveraging Parameter Space Symmetries for Reasoning Skill Transfer in LLMs. Come say hi! Would love to connect and exchange ideas Paper: arxiv.org/abs/2511.10850

SEACrowd avatar
⟳ Genta Winata retweeted
SEACrowd@seacrowd_ai·30 Nov 2025

🌏 Applications are now open for the SEACrowd Apprentice Program 2026! Join a 3–4 month guided research journey where you’ll collaborate with mentors. 🗓️ Apply between Nov 17 – Dec 17, 2025 (UTC-12) 📅 Program runs Feb – Jun 2026 seacrowd.org/apprenticeship

David Anugraha avatar
⟳ Genta Winata retweeted
David Anugraha@DavidAnugraha·6 Oct 2025

🚀 Following R3, we are thrilled to introduce mR3: Multilingual Rubric-Agnostic Reward Reasoning Models 🎉 TL;DR: mR3 is a multilingual version of R3, a rubric-agnostic reward reasoning model trained on 72 languages, achieving the broadest language coverage in reward modeling to date. mR3 is able to surpass much larger models, such as GPT-OSS-120B, while being up to 9x smaller⚡! 📄 Paper: arxiv.org/abs/2510.01146 💻 Code: github.com/rubricreward/mr3 🤗 mR3 Models: huggingface.co/collections/r… 🤗 mR3 Datasets: huggingface.co/collections/r… Huge thanks to all of my collaborators: @HLeiTR, @Zilu_Tang_Peter, @ Annie Lee, @DerryTWijaya1, @llamagrp, and @gentaiscool for all of their efforts into creating this work! Check this thread out! #mR3 #RewardModel #NLP #LLM #Metrics #Alignment #Preference (1/4)

Genta Winata avatar
Genta Winata@gentaiscool·18 Sep 2025

Key Highlights: - 𝐓𝟏, a comprehensive multi-turn dataset consisting of 13.5k dialogues designed to evaluate tool using, LLM-based agents across nine key domains. The dataset covers a wide range of interaction scenarios, including single-domain, mixed-domain, and fully multi-domain conversations. It incorporates 14 distinct tools, enabling realistic and fine-grained assessment of agent capabilities in complex, tool-driven dialogue tasks. - Cross-domain tasks and interdependent tool calls within the dataset to increase complexity and realism of the dataset, requiring agents to reason about tool selection and execution order within context. - 𝐓𝟏-𝐀𝐆𝐄𝐍𝐓, An LLM based code-generation system, built on open-source language models and equipped with a caching mechanism for improved performance. This architecture enables scalable evaluation and provides a robust, efficient framework for tool-using agents.