Evaluation

On the Limitations of Simulating Active Learning ACL 2023 On Tilted Losses in Machine Learning: Theory and Applications JMLR 2023 Ontology-aware Learning and Evaluation for Audio Tagging Interspeech 2023 OpenDataVal: a Unified Benchmark for Data Valuation NeurIPS 2023 Optimizing ROC Curves with a Sort-Based Surrogate Loss for Binary Classification and Changepoint Detection JMLR 2023 ORCA: A Challenging Benchmark for Arabic Language Understanding ACL 2023 Post Turing: Mapping the landscape of LLM Evaluation EMNLP 2023 PragmatiCQA: A Dataset for Pragmatic Question Answering in Conversations ACL 2023 Precision Recall Cover: A Method For Assessing Generative Models AISTATS 2023 Precision/Recall on Imbalanced Test Data AISTATS 2023 Predicting Question-Answering Performance of Large Language Models through Semantic Consistency EMNLP 2023 Primacy Effect of ChatGPT EMNLP 2023 PriMeSRL-Eval: A Practical Quality Metric for Semantic Role Labeling Systems Evaluation EACL 2023 Probing the “Creativity” of Large Language Models: Can models produce divergent semantic association? EMNLP 2023 Pulling Out All The Full Stops: Punctuation Sensitivity in Neural Machine Translation and Evaluation ACL 2023 Quality Estimation Using Minimum Bayes Risk EMNLP 2023 Range-Based Equal Error Rate for Spoof Localization Interspeech 2023 Ranger: A Toolkit for Effect-Size Based Multi-Task Evaluation ACL 2023 Rarely a problem? Language models exhibit inverse scaling in their predictions following few-type quantifiers ACL 2023 READIN: A Chinese Multi-Task Benchmark with Realistic and Diverse Input Noises ACL 2023 Reasoning about Ambiguous Definite Descriptions EMNLP 2023 Re-Examining Summarization Evaluation across Multiple Quality Criteria EMNLP 2023 Regions of Reliability in the Evaluation of Multivariate Probabilistic Forecasts ICML 2023 Reproducibility in NLP: What Have We Learned from the Checklist? ACL 2023 Responsible AI Considerations in Text Summarization Research: A Review of Current Practices EMNLP 2023