Evaluation

Subject Verb Agreement Error Patterns in Meaningless Sentences: Humans vs. BERT COLING 2022 SueNes: A Weakly Supervised Approach to Evaluating Single-Document Summarization via Negative Sampling NAACL 2022 SummScreen: A Dataset for Abstractive Screenplay Summarization ACL 2022 Systematicity, Compositionality and Transitivity of Deep NLP Models: a Metamorphic Testing Perspective ACL 2022 TestAug: A Framework for Augmenting Capability-based NLP Tests COLING 2022 Testing Large Language Models on Compositionality and Inference with Phrase-Level Adjective-Noun Entailment COLING 2022 Testing the Ability of Language Models to Interpret Figurative Language NAACL 2022 Test Set Sampling Affects System Rankings: Expanded Human Evaluation of WMT20 English-Inuktitut Systems EMNLP 2022 Texture-Based Error Analysis for Image Super-Resolution CVPR 2022 The Change that Matters in Discourse Parsing: Estimating the Impact of Domain Shift on Parser Error ACL 2022 The CLRS Algorithmic Reasoning Benchmark ICML 2022 The King Is Naked: On the Notion of Robustness for Natural Language Processing AAAI 2022 The patient is more dead than alive: exploring the current state of the multi-document summarisation of the biomedical literature ACL 2022 The Role of Context and Uncertainty in Shallow Discourse Parsing COLING 2022 This is the way: designing and compiling LEPISZCZE, a comprehensive NLP benchmark for Polish NeurIPS 2022 Tomayto, Tomahto. Beyond Token-level Answer Equivalence for Question Answering Evaluation EMNLP 2022 Toward More Effective Human Evaluation for Machine Translation ACL 2022 Towards a Rigorous Evaluation of Time-Series Anomaly Detection AAAI 2022 Towards a Unified Multi-Dimensional Evaluator for Text Generation EMNLP 2022 Towards Driving-Oriented Metric for Lane Detection Models CVPR 2022 Towards Explainable Evaluation of Language Models on the Semantic Similarity of Visual Concepts COLING 2022 Towards Robust NLG Bias Evaluation with Syntactically-diverse Prompts EMNLP 2022 To What Extent Do Natural Language Understanding Datasets Correlate to Logical Reasoning? A Method for Diagnosing Logical Reasoning. COLING 2022 Transparent Human Evaluation for Image Captioning NAACL 2022 TruthfulQA: Measuring How Models Mimic Human Falsehoods ACL 2022