Evaluation

Generalized Quantifiers as a Source of Error in Multilingual NLU Benchmarks NAACL 2022 Generic Overgeneralization in Pre-trained Language Models COLING 2022 GENIE: Toward Reproducible and Standardized Human Evaluation for Text Generation EMNLP 2022 Geographic Citation Gaps in NLP Research EMNLP 2022 Hard and Soft Evaluation of NLP models with BOOtSTrap SAmpling - BooStSa ACL 2022 How Conservative are Language Models? Adapting to the Introduction of Gender-Neutral Pronouns NAACL 2022 How Did the Model Change? Efficiently Assessing Machine Learning API Shifts ICLR 2022 How Language-Dependent is Emotion Detection? Evidence from Multilingual BERT EMNLP 2022 How Large Language Models are Transforming Machine-Paraphrase Plagiarism EMNLP 2022 How Much Does Attention Actually Attend? Questioning the Importance of Attention in Pretrained Transformers EMNLP 2022 Human Evaluation and Correlation with Automatic Metrics in Consultation Note Generation ACL 2022 Human Evaluation of Conversations is an Open Problem: comparing the sensitivity of various methods for evaluating dialogue agents ACL 2022 Human evaluation of web-crawled parallel corpora for machine translation ACL 2022 Human perceiving behavior modeling in evaluation of code generation models EMNLP 2022 Identifying Weaknesses in Machine Translation Metrics Through Minimum Bayes Risk Decoding: A Case Study for COMET AACL 2022 ILDAE: Instance-Level Difficulty Analysis of Evaluation Data ACL 2022 IM2: an Interpretable and Multi-category Integrated Metric Framework for Automatic Dialogue Evaluation EMNLP 2022 Imbalance-Aware Uplift Modeling for Observational Data AAAI 2022 Impact of Pretraining Term Frequencies on Few-Shot Numerical Reasoning EMNLP 2022 IMPARA: Impact-Based Metric for GEC Using Parallel Data COLING 2022 InfoLM: A New Metric to Evaluate Summarization & Data2Text Generation AAAI 2022 InteractEva: A Simulation-Based Evaluation Framework for Interactive AI Systems AAAI 2022 Inter-annotator agreement is not the ceiling of machine learning performance: Evidence from a comprehensive set of simulations ACL 2022 Investigating Crowdsourcing Protocols for Evaluating the Factual Consistency of Summaries NAACL 2022 Investigating Data Variance in Evaluations of Automatic Machine Translation Metrics ACL 2022