Evaluation

Is Summary Useful or Not? An Extrinsic Human Evaluation of Text Summaries on Downstream Tasks COLING 2024 JCoLA: Japanese Corpus of Linguistic Acceptability COLING 2024 JobFair: A Framework for Benchmarking Gender Hiring Bias in Large Language Models EMNLP 2024 kGym: A Platform and Dataset to Benchmark Large Language Models on Linux Kernel Crash Resolution NeurIPS 2024 KIEval: A Knowledge-grounded Interactive Evaluation Framework for Large Language Models ACL 2024 KoDialogBench: Evaluating Conversational Understanding of Language Models with Korean Dialogue Benchmark COLING 2024 KPEval: Towards Fine-Grained Semantic-Based Keyphrase Evaluation ACL 2024 Language Models can Evaluate Themselves via Probability Discrepancy ACL 2024 Language Models Do Hard Arithmetic Tasks Easily and Hardly Do Easy Arithmetic Tasks ACL 2024 Large Language Model as an Assignment Evaluator: Insights, Feedback, and Challenges in a 1000+ Student Course EMNLP 2024 Large Language Models are not Fair Evaluators ACL 2024 Large Language Models' Expert-level Global History Knowledge Benchmark (HiST-LLM) NeurIPS 2024 Large Language Models Fall Short: Understanding Complex Relationships in Detective Narratives ACL 2024 LatestEval: Addressing Data Contamination in Language Model Evaluation through Dynamic and Time-Sensitive Test Construction AAAI 2024 Latxa: An Open Language Model and Evaluation Suite for Basque ACL 2024 LawBench: Benchmarking Legal Knowledge of Large Language Models EMNLP 2024 LEGOBench: Scientific Leaderboard Generation Benchmark EMNLP 2024 Liar, Liar, Logical Mire: A Benchmark for Suppositional Reasoning in Large Language Models EMNLP 2024 Likelihood-based Mitigation of Evaluation Bias in Large Language Models ACL 2024 LJPCheck: Functional Tests for Legal Judgment Prediction ACL 2024 LLM Evaluators Recognize and Favor Their Own Generations NeurIPS 2024 LLM Task Interference: An Initial Study on the Impact of Task-Switch in Conversational History EMNLP 2024 Localization Is All You Evaluate: Data Leakage in Online Mapping Datasets and How to Fix It CVPR 2024 LongDocFACTScore: Evaluating the Factuality of Long Document Abstractive Summarisation COLING 2024 Long-form factuality in large language models NeurIPS 2024