Evaluation

We Need To Talk About Random Splits EACL 2021 We Need to Talk About train-dev-test Splits EMNLP 2021 What Does Rotation Prediction Tell Us about Classifier Accuracy under Varying Testing Environments? ICML 2021 What Ingredients Make for an Effective Crowdsourcing Protocol for Difficult NLU Data Collection Tasks? ACL 2021 What is SemEval evaluating? A Systematic Analysis of Evaluation Campaigns in NLP EMNLP 2021 What Makes a Scientific Paper be Accepted for Publication? EMNLP 2021 When All We Need is a Piece of the Pie: A Generic Framework for Optimizing Two-way Partial AUC ICML 2021 XTREME-R: Towards More Challenging and Nuanced Multilingual Evaluation EMNLP 2021 YASO: A Targeted Sentiment Analysis Evaluation Dataset for Open-Domain Reviews EMNLP 2021 Abstract Interpretation of Decision Tree Ensemble Classifiers AAAI 2020 Active Testing: An Unbiased Evaluation Method for Distantly Supervised Relation Extraction EMNLP 2020 A Diverse Corpus for Evaluating and Developing English Math Word Problem Solvers ACL 2020 Adversarial NLI: A New Benchmark for Natural Language Understanding ACL 2020 A Methodology for Creating Question Answering Corpora Using Inverse Data Annotation ACL 2020 Analyzing Neural Discourse Coherence Models EMNLP 2020 An Analysis of Natural Language Inference Benchmarks through the Lens of Negation EMNLP 2020 An Effectiveness Metric for Ordinal Classification: Formal Properties and Experimental Results ACL 2020 An Empirical Analysis of Human-Bot Interaction on Reddit EMNLP 2020 Approximate Cross-Validation for Structured Models NeurIPS 2020 A Re-evaluation of Knowledge Graph Completion Methods ACL 2020 A Report on the VarDial Evaluation Campaign 2020 COLING 2020 Are Some Words Worth More than Others? EMNLP 2020 Are we Estimating or Guesstimating Translation Quality? ACL 2020 Assessing Human-Parity in Machine Translation on the Segment Level EMNLP 2020 Assessing the Benchmarking Capacity of Machine Reading Comprehension Datasets AAAI 2020