Evaluation

Large Language Models Badly Generalize across Option Length, Problem Types, and Irrelevant Noun Replacements EMNLP 2025 L-CiteEval: A Suite for Evaluating Fidelity of Long-context Models ACL 2025 LIFBench: Evaluating the Instruction Following Performance and Stability of Large Language Models in Long-Context Scenarios ACL 2025 LLM-as-an-Interviewer: Beyond Static Testing Through Dynamic LLM Evaluation ACL 2025 LLMs can be easily Confused by Instructional Distractions ACL 2025 LLM The Genius Paradox: A Linguistic and Math Expert’s Struggle with Simple Word-based Counting Problems NAACL 2025 LR²Bench: Evaluating Long-chain Reflective Reasoning Capabilities of Large Language Models via Constraint Satisfaction Problems ACL 2025 McBE: A Multi-task Chinese Bias Evaluation Benchmark for Large Language Models ACL 2025 Metric Calculating Benchmark: Code-Verifiable Complicate Instruction Following Benchmark for Large Language Models EMNLP 2025 Mind the Gap: A Closer Look at Tokenization for Multiple-Choice Question Answering with LLMs EMNLP 2025 Mind the Gap: Static and Interactive Evaluations of Large Audio Models ACL 2025 MLDebugging: Towards Benchmarking Code Debugging Across Multi-Library Scenarios ACL 2025 MMRefine: Unveiling the Obstacles to Robust Refinement in Multimodal Large Language Models ACL 2025 Model Consistency as a Cheap yet Predictive Proxy for LLM Elo Scores EMNLP 2025 M-RewardBench: Evaluating Reward Models in Multilingual Settings ACL 2025 NESTFUL: A Benchmark for Evaluating LLMs on Nested Sequences of API Calls EMNLP 2025 NormXLogit: The Head-on-Top Never Lies EMNLP 2025 ODE: Open-Set Evaluation of Hallucinations in Multimodal Large Language Models CVPR 2025 Open Universal Arabic ASR Leaderboard Interspeech 2025 Optimization before Evaluation: Evaluation with Unoptimized Prompts Can be Misleading ACL 2025 PCRI: Measuring Context Robustness in Multimodal Models for Enterprise Applications EMNLP 2025 Persona-Augmented Benchmarking: Evaluating LLMs Across Diverse Writing Styles EMNLP 2025 Praetor: A Fine-Grained Generative LLM Evaluator with Instance-Level Customizable Evaluation Criteria ACL 2025 Probing the Mid-level Vision Capabilities of Self-Supervised Learning CVPR 2025 Pun Unintended: LLMs and the Illusion of Humor Understanding EMNLP 2025