multilingual benchmark

XCOMPS: A Multilingual Benchmark of Conceptual Minimal Pairs ACL 2025 Expanding the FLORES+ Multilingual Benchmark with Translations for Aragonese, Aranese, Asturian, and Valencian EMNLP 2024 IndicGenBench: A Multilingual Benchmark to Evaluate Generation Capabilities of LLMs on Indic Languages ACL 2024 Investigating the Potential of Task Arithmetic for Cross-Lingual Transfer EACL 2024 Lost in Translation? Translation Errors and Challenges for Fair Assessment of Text-to-Image Models on Multilingual Concepts NAACL 2024 M4GT-Bench: Evaluation Benchmark for Black-Box Machine-Generated Text Detection ACL 2024 MLissard: Multilingual Long and Simple Sequential Reasoning Benchmarks EMNLP 2024 SciEx: Benchmarking Large Language Models on Scientific Exams with Human Expert Grading and Automatic Grading EMNLP 2024 The Belebele Benchmark: a Parallel Reading Comprehension Dataset in 122 Language Variants ACL 2024 Universal NER: A Gold-Standard Multilingual Named Entity Recognition Benchmark NAACL 2024 XCodeEval: An Execution-based Large Scale Multilingual Multitask Benchmark for Code Understanding, Generation, Translation and Retrieval ACL 2024 CodeTransOcean: A Comprehensive Multilingual Benchmark for Code Translation EMNLP 2023 DLAMA: A Framework for Curating Culturally Diverse Facts for Probing the Knowledge of Pretrained Language Models ACL 2023 Do All Languages Cost the Same? Tokenization in the Era of Commercial Language Models EMNLP 2023 ML-SUPERB: Multilingual Speech Universal PERformance Benchmark Interspeech 2023 Revisiting non-English Text Simplification: A Unified Multilingual Benchmark ACL 2023 The SIGMORPHON 2022 Shared Task on Cross-lingual and Low-Resource Grapheme-to-Phoneme Conversion ACL 2023 When Does Translation Require Context? A Data-driven, Multilingual Exploration ACL 2023 XL-LEXEME: WiC Pretrained Model for Cross-Lingual LEXical sEMantic changE ACL 2023 A Multilingual Benchmark to Capture Olfactory Situations over Time ACL 2022 Beyond Static models and test sets: Benchmarking the potential of pre-trained models across tasks and languages ACL 2022 Benchmarking: Past, Present and Future ACL 2021 IndoNLG: Benchmark and Resources for Evaluating Indonesian Natural Language Generation EMNLP 2021 Results of the Second SIGMORPHON Shared Task on Multilingual Grapheme-to-Phoneme Conversion ACL 2021 XTREME-R: Towards More Challenging and Nuanced Multilingual Evaluation EMNLP 2021