Evaluation

LOTUS: A Leaderboard for Detailed Image Captioning from Quality to Societal Bias and User Preferences ACL 2025 LSC-Eval: A General Framework to Evaluate Methods for Assessing Dimensions of Lexical Semantic Change Using LLM-Generated Synthetic Data ACL 2025 L-SWAG: Layer-Sample Wise Activation with Gradients Information for Zero-Shot NAS on Vision Transformers CVPR 2025 MALAMUTE: A Multilingual, Highly-granular, Template-free, Education-based Probing Dataset ACL 2025 MapQaTor: An Extensible Framework for Efficient Annotation of Map-Based QA Datasets ACL 2025 MCQFormatBench: Robustness Tests for Multiple-Choice Questions ACL 2025 MDBench: A Synthetic Multi-Document Reasoning Benchmark Generated with Knowledge Guidance ACL 2025 Measuring Bias and Agreement in Large Language Model Presupposition Judgments ACL 2025 Measuring Data Diversity for Instruction Tuning: A Systematic Analysis and A Reliable Metric ACL 2025 Measuring Mental Health Variables in Computational Research: Toward Validated, Dimensional, and Transdiagnostic Approaches NAACL 2025 Measuring scalar constructs in social science with LLMs EMNLP 2025 Mechanisms vs. Outcomes: Probing for Syntax Fails to Explain Performance on Targeted Syntactic Evaluations EMNLP 2025 MedFact: A Large-scale Chinese Dataset for Evidence-based Medical Fact-checking of LLM Responses EMNLP 2025 Membership Inference Attacks with False Discovery Rate Control ICCV 2025 MEMERAG: A Multilingual End-to-End Meta-Evaluation Benchmark for Retrieval Augmented Generation ACL 2025 Memorization or Reasoning? Exploring the Idiom Understanding of LLMs EMNLP 2025 Memorization ≠ Understanding: Do Large Language Models Have the Ability of Scenario Cognition? EMNLP 2025 Metric Calculating Benchmark: Code-Verifiable Complicate Instruction Following Benchmark for Large Language Models EMNLP 2025 MFTCXplain: A Multilingual Benchmark Dataset for Evaluating the Moral Reasoning of LLMs through Multi-hop Hate Speech Explanation EMNLP 2025 M-IFEval: Multilingual Instruction-Following Evaluation NAACL 2025 Mind the Query: A Benchmark Dataset towards Text2Cypher Task EMNLP 2025 MiniLongBench: The Low-cost Long Context Understanding Benchmark for Large Language Models ACL 2025 Minimal Pair-Based Evaluation of Code-Switching ACL 2025 MisinfoBench: A Multi-Dimensional Benchmark for Evaluating LLMs’ Resilience to Misinformation EMNLP 2025 Mis-prompt: Benchmarking Large Language Models for Proactive Error Handling ACL 2025