
11 AM GST: Panel Discussion-1: Challenges of Collecting Culturally Grounded Multilingual Data for Training and Evaluation of NLP Systems.

2nd Workshop on Scaling Up Multilingual & Multi-Cultural Evaluation @ COLING 2025

11 AM GST: Panel Discussion-1: Challenges of Collecting Culturally Grounded Multilingual Data for Training and Evaluation of NLP Systems.

SumEval-2 is happening today at ADNEC, Abu Dhabhi. Looking forward to meeting all the participants!

**Archival Submission Deadline** Reminder to submit your archival submissions by Nov 8th, 2024 (AOE) Submission Link: softconf.com/coling2025/SUME… Please check our website for more details: sites.google.com/view/sumeva… or contact us at sumeval2025@gmail.com

Softconf Submission Link: softconf.com/coling2025/SUME… Please check our website for more details: lnkd.in/gqb--HmT or contact us at sumeval2025@gmail.com

Thanks to Microsoft for their generous sponsorship for their cash awards.

** Updates ** * We are excited to announce that, there will be cash awards for two best papers, one for archival and one for non-archival track * We are extending the archival submission deadline to Nov 8, 2024 (no changes to the non-archival submission deadline Nov 25, 2024)

with: @monojitchou , @vishrav , Sunayana Sitaram, @seb_ruder , @HellinaNigatu , @OanaIgnatRo

***SumEval Archival Submission Deadline*** Archival Submission: October 18th, 2024 Softconf Submission Link: softconf.com/coling2025/SUME… Please check our website for more details: sites.google.com/view/sumeva…… or contact us at sumeval2025@gmail.com

Please check our website for more details: sites.google.com/view/sumeva… or contact us at sumeval2025@gmail.com

***Important Dates*** Archival Submission: October 18th, 2024 Non-archival Submission: November 25th, 2024 Notification of Acceptance: December 3rd, 2024 Camera-Ready Papers Due: December 10th, 2024 Workshop Date: January 20th, 2025, co-located with COLING 2025

* Probing and analysis of multilingual models across cultures and languages * Comparison of evaluation strategies for languages, cultures, and domains

* Automated evaluation metrics for multilingual and multicultural evaluation * Studies on fairness and other aspects of evaluation * Studies on cultural representation in multilingual models * Data sets, benchmarks, or libraries for evaluating multi-lingual models

We are inviting submissions on the following (and more) topics: * Under-represented cultures and languages * Studies on scaling up multilingual and multicultural evaluation * Human evaluation of multilingual and multicultural aspects of models

Announcing the second edition of SUMEval - Workshop on Scaling Up Multilingual and Multi-Cultural Evaluation of foundation models will be held in conjunction with COLING 2025. with : @HellinaNigatu , @Oanaignat, @seb_ruder, Sunayana Sitaram, @monojitchou, @vishrav

In addition to regular papers submitted to the workshop, we will also accept papers that were submitted elsewhere and papers with AACL and ARR reviews. For detailed information regarding submission instructions, please visit lnkd.in/g57vnuWC

** SumEVAL Paper Submission Deadline Extended ** We are extending the SumEVAL workshop paper submission deadline to September 23, 2022.