Evaluation

GENTLE: A Genre-Diverse Multilayer Challenge Set for English NLP and Linguistic Evaluation ACL 2023 Give Me the Facts! A Survey on Factual Knowledge Probing in Pre-trained Language Models EMNLP 2023 GlobalBench: A Benchmark for Global Progress in Natural Language Processing EMNLP 2023 GLUE-X: Evaluating Natural Language Understanding Models from an Out-of-Distribution Generalization Perspective ACL 2023 GNNEvaluator: Evaluating GNN Performance On Unseen Graphs Without Labels NeurIPS 2023 GSLB: The Graph Structure Learning Benchmark NeurIPS 2023 Harmful Language Datasets: An Assessment of Robustness ACL 2023 HAUSER: Towards Holistic and Automatic Evaluation of Simile Generation ACL 2023 Holistic Inter-Annotator Agreement and Corpus Coherence Estimation in a Large-scale Multilingual Annotation Campaign EMNLP 2023 How Many Perturbations Break This Model? Evaluating Robustness Beyond Adversarial Accuracy ICML 2023 How Predictable Are Large Language Model Capabilities? A Case Study on BIG-bench EMNLP 2023 How to Construct Perfect and Worse-than-Coin-Flip Spoofing Countermeasures: A Word of Warning on Shortcut Learning Interspeech 2023 How to Determine the Most Powerful Pre-trained Language Model without Brute Force Fine-tuning? An Empirical Survey EMNLP 2023 How Well Do Large Language Models Perform on Faux Pas Tests? ACL 2023 How Well Do Text Embedding Models Understand Syntax? EMNLP 2023 HRS-Bench: Holistic, Reliable and Scalable Benchmark for Text-to-Image Models ICCV 2023 IAEval: A Comprehensive Evaluation of Instance Attribution on Natural Language Understanding EMNLP 2023 Improved Policy Evaluation for Randomized Trials of Algorithmic Resource Allocation ICML 2023 Incorporating Attribution Importance for Improving Faithfulness Metrics ACL 2023 Influence Diagnostics under Self-concordance AISTATS 2023 Influence Scores at Scale for Efficient Language Data Sampling EMNLP 2023 Information Extraction from Legal Wills: How Well Does GPT-4 Do? EMNLP 2023 Interview Evaluation: A Novel Approach for Automatic Evaluation of Conversational Question Answering Models EMNLP 2023 Investigating Reproducibility at Interspeech Conferences: A Longitudinal and Comparative Perspective Interspeech 2023 Investigating UD Treebanks via Dataset Difficulty Measures EACL 2023