benchmark evaluation

CodaLab Competitions: An Open Source Platform to Organize Scientific Challenges JMLR 2023 CompleQA: Benchmarking the Impacts of Knowledge Graph Completion Methods on Question Answering EMNLP 2023 CRAB: Assessing the Strength of Causal Relationships Between Real-world Events EMNLP 2023 CREPE: Can Vision-Language Foundation Models Reason Compositionally? CVPR 2023 CrossCodeEval: A Diverse and Multilingual Benchmark for Cross-File Code Completion NeurIPS 2023 Detecting Edit Failures In Large Language Models: An Improved Specificity Benchmark ACL 2023 DFEE: Interactive DataFlow Execution and Evaluation Kit AAAI 2023 Distinguishing Cause and Effect in Bivariate Structural Causal Models: A Systematic Investigation JMLR 2023 Do Androids Laugh at Electric Sheep? Humor “Understanding” Benchmarks from The New Yorker Caption Contest ACL 2023 Does Continual Learning Meet Compositionality? New Benchmarks and An Evaluation Framework NeurIPS 2023 Do Language Models Have a Common Sense regarding Time? Revisiting Temporal Commonsense Reasoning in the Era of Large Language Models EMNLP 2023 Do Question Answering Modeling Improvements Hold Across Benchmarks? ACL 2023 DS-1000: A Natural and Reliable Benchmark for Data Science Code Generation ICML 2023 Elo Uncovered: Robustness and Best Practices in Language Model Evaluation EMNLP 2023 Empathy Identification Systems are not Accurately Accounting for Context EACL 2023 EpiK-Eval: Evaluation for Language Models as Epistemic Models EMNLP 2023 Evaluating context-invariance in unsupervised speech representations Interspeech 2023 Evaluating Cross-Domain Text-to-SQL Models and Benchmarks EMNLP 2023 Evaluating Large Language Models on Controlled Generation Tasks EMNLP 2023 Evaluating Neural Language Models as Cognitive Models of Language Acquisition EMNLP 2023 Evaluating the Factual Consistency of Large Language Models Through News Summarization ACL 2023 Exploring the Capacity of Pretrained Language Models for Reasoning about Actions and Change ACL 2023 FANToM: A Benchmark for Stress-testing Machine Theory of Mind in Interactions EMNLP 2023 FERMAT: An Alternative to Accuracy for Numerical Reasoning ACL 2023 Fighting FIRe with FIRE: Assessing the Validity of Text-to-Video Retrieval Benchmarks EACL 2023