evaluation methodology

Benchmark Data Repositories for Better Benchmarking NeurIPS 2024 Beyond Probabilities: Unveiling the Misalignment in Evaluating Large Language Models ACL 2024 Domain Dynamics: Evaluating Large Language Models in English-Hindi Translation EMNLP 2024 Enough Is Enough! a Case Study on the Effect of Data Size for Evaluation Using Universal Dependencies COLING 2024 Evaluation and Continual Improvement for an Enterprise AI Assistant NAACL 2024 Human-Centered Evaluation of Language Technologies EMNLP 2024 Medical Dialogue System: A Survey of Categories, Methods, Evaluation and Challenges ACL 2024 “My Answer is C”: First-Token Probabilities Do Not Match Text Answers in Instruction-Tuned Language Models ACL 2024 ReproHum #0124-03: Reproducing Human Evaluations of end-to-end approaches for Referring Expression Generation COLING 2024 Rethinking Evaluation Methods for Machine Unlearning EMNLP 2024 The 2024 ReproNLP Shared Task on Reproducibility of Evaluations in NLP: Overview and Results COLING 2024 The Art of Defending: A Systematic Evaluation and Analysis of LLM Defense Strategies on Safety and Over-Defensiveness ACL 2024 The Base-Rate Effect on LLM Benchmark Performance: Disambiguating Test-Taking Strategies from Benchmark Performance EMNLP 2024 The GenderQueer Test Suite EMNLP 2024 A Call for Standardization and Validation of Text Style Transfer Evaluation ACL 2023 Evaluating Zero-Shot Event Structures: Recommendations for Automatic Content Extraction (ACE) Annotations ACL 2023 Evaluation for Change ACL 2023 Investigating Range-Equalizing Bias in Mean Opinion Score Ratings of Synthesized Speech Interspeech 2023 Missing Information, Unresponsive Authors, Experimental Flaws: The Impossibility of Assessing the Reproducibility of Previous Human Evaluations in NLP EACL 2023 Morphological Inflection: A Reality Check ACL 2023 Navigating the Pitfalls of Active Learning Evaluation: A Systematic Framework for Meaningful Performance Assessment NeurIPS 2023 On Degrees of Freedom in Defining and Testing Natural Language Understanding ACL 2023 Post Turing: Mapping the landscape of LLM Evaluation EMNLP 2023 Reduce Human Labor On Evaluating Conversational Information Retrieval System: A Human-Machine Collaboration Approach EMNLP 2023 Rogue Scores ACL 2023