Evaluation

Line of Duty: Evaluating LLM Self-Knowledge via Consistency in Feasibility Boundaries NAACL 2025 Linguistic Alignment Predicts Learning in Small Group Tutoring Sessions EMNLP 2025 LiteraryQA: Towards Effective Evaluation of Long-document Narrative QA EMNLP 2025 LLM-as-an-Interviewer: Beyond Static Testing Through Dynamic LLM Evaluation ACL 2025 LLM-based post-editing as reference-free GEC evaluation ACL 2025 LLM Evaluate: An Industry-Focused Evaluation Tool for Large Language Models COLING 2025 LLMs are Biased Evaluators But Not Biased for Fact-Centric Retrieval Augmented Generation ACL 2025 LLMs Are Biased Towards Output Formats! Systematically Evaluating and Mitigating Output Format Bias of LLMs NAACL 2025 LLMs Are Not Intelligent Thinkers: Introducing Mathematical Topic Tree Benchmark for Comprehensive Evaluation of LLMs NAACL 2025 LLMs as Medical Safety Judges: Evaluating Alignment with Human Annotation in Patient-Facing QA ACL 2025 LLMs can be easily Confused by Instructional Distractions ACL 2025 LLMs cannot spot math errors, even when allowed to peek into the solution EMNLP 2025 LLMs instead of Human Judges? A Large Scale Empirical Study across 20 NLP Evaluation Tasks ACL 2025 LLM The Genius Paradox: A Linguistic and Math Expert’s Struggle with Simple Word-based Counting Problems NAACL 2025 LoCt-Instruct: An Automatic Pipeline for Constructing Datasets of Logical Continuous Instructions EMNLP 2025 LOFT: Scalable and More Realistic Long-Context Evaluation NAACL 2025 Logical forms complement probability in understanding language model (and human) performance ACL 2025 LogicGame: Benchmarking Rule-Based Reasoning Abilities of Large Language Models ACL 2025 LogiDynamics: Unraveling the Dynamics of Inductive, Abductive and Deductive Logical Inferences in LLM Reasoning EMNLP 2025 Long-context Reference-based MT Quality Estimation EMNLP 2025 Long-Form Information Alignment Evaluation Beyond Atomic Facts EMNLP 2025 LongTableBench: Benchmarking Long-Context Table Reasoning across Real-World Formats and Domains EMNLP 2025 LongTail-Swap: benchmarking language models’ abilities on rare words EMNLP 2025 LongWeave: A Long-Form Generation Benchmark Bridging Real-World Relevance and Verifiability EMNLP 2025 Lost in Transcription, Found in Distribution Shift: Demystifying Hallucination in Speech Foundation Models ACL 2025