benchmark evaluation

TAPE: Assessing Few-shot Russian Language Understanding EMNLP 2022 TempoWiC: An Evaluation Benchmark for Detecting Meaning Shift in Social Media COLING 2022 The DEVIL Is in the Details: A Diagnostic Evaluation Benchmark for Video Inpainting CVPR 2022 The Paradox of the Compositionality of Natural Language: A Neural Machine Translation Case Study ACL 2022 This is the way: designing and compiling LEPISZCZE, a comprehensive NLP benchmark for Polish NeurIPS 2022 Towards a Rigorous Evaluation of Time-Series Anomaly Detection AAAI 2022 Towards a Standardised Performance Evaluation Protocol for Cooperative MARL NeurIPS 2022 VALSE: A Task-Independent Benchmark for Vision and Language Models Centered on Linguistic Phenomena ACL 2022 VALUE: Understanding Dialect Disparity in NLU ACL 2022 VLUE: A Multi-Task Multi-Dimension Benchmark for Evaluating Vision-Language Pre-training ICML 2022 What are the best Systems? New Perspectives on NLP Benchmarking NeurIPS 2022 What Makes Reading Comprehension Questions Difficult? ACL 2022 What’s Different between Visual Question Answering for Machine “Understanding” Versus for Accessibility? AACL 2022 XTREME-S: Evaluating Cross-lingual Speech Representations Interspeech 2022 Are NLP Models really able to Solve Simple Math Word Problems? NAACL 2021 Benchmarking: Past, Present and Future ACL 2021 Beware of the Simulated DAG! Causal Discovery Benchmarks May Be Easy to Game NeurIPS 2021 Beyond the Tip of the Iceberg: Assessing Coherence of Text Classifiers EMNLP 2021 BREEDS: Benchmarks for Subpopulation Shift ICLR 2021 Characterizing Generalization under Out-Of-Distribution Shifts in Deep Metric Learning NeurIPS 2021 Concept Generalization in Visual Representation Learning ICCV 2021 Deep Reinforcement Learning at the Edge of the Statistical Precipice NeurIPS 2021 Descending through a Crowded Valley - Benchmarking Deep Learning Optimizers ICML 2021 Developing a Benchmark for Reducing Data Bias in Authorship Attribution EMNLP 2021 Diagnosing the First-Order Logical Reasoning Ability Through LogicNLI EMNLP 2021