Evaluation

ImageNet-X: Understanding Model Mistakes with Factor of Variation Annotations ICLR 2023 NeuroEvoBench: Benchmarking Evolutionary Optimizers for Deep Learning Applications NeurIPS 2023 POPGym: Benchmarking Partially Observable Reinforcement Learning ICLR 2023 Task Ambiguity in Humans and Language Models ICLR 2023 The Devil is in the Details: On the Pitfalls of Event Extraction Evaluation ACL 2023 VisoGender: A dataset for benchmarking gender bias in image-text pronoun resolution NeurIPS 2023 ADBench: Anomaly Detection Benchmark NeurIPS 2022 BenchIE: A Framework for Multi-Faceted Fact-Based Open Information Extraction Evaluation ACL 2022 Benchmarking the Spectrum of Agent Capabilities ICLR 2022 Evaluating Distributional Distortion in Neural Language Modeling ICLR 2022 miniF2F: a cross-system benchmark for formal Olympiad-level mathematics ICLR 2022 The MultiBERTs: BERT Reproductions for Robustness Analysis ICLR 2022 Aligning AI With Shared Human Values ICLR 2021 Baby Intuitions Benchmark (BIB): Discerning the goals, preferences, and actions of others NeurIPS 2021 Benchmarks for Deep Off-Policy Evaluation ICLR 2021 CoCo: Controllable Counterfactuals for Evaluating Dialogue State Trackers ICLR 2021 In Search of Lost Domain Generalization ICLR 2021 Long Range Arena : A Benchmark for Efficient Transformers ICLR 2021 Measuring Massive Multitask Language Understanding ICLR 2021 TabPert : An Effective Platform for Tabular Perturbation EMNLP 2021 Towards Automatic Evaluation of Dialog Systems: A Model-Free Off-Policy Evaluation Approach EMNLP 2021 BERTScore: Evaluating Text Generation with BERT ICLR 2020 Measuring Compositional Generalization: A Comprehensive Method on Realistic Data ICLR 2020 Theory and Evaluation Metrics for Learning Disentangled Representations ICLR 2020 Can you trust your model's uncertainty? Evaluating predictive uncertainty under dataset shift NeurIPS 2019