benchmark evaluation

PIPER: Benchmarking and Prompting Event Reasoning Boundary of LLMs via Debiasing-Distillation Enhanced Tuning ACL 2025 Planetarium: A Rigorous Benchmark for Translating Text to Structured Planning Languages NAACL 2025 PlanningArena: A Modular Benchmark for Multidimensional Evaluation of Planning and Tool Learning ACL 2025 PL-Guard: Benchmarking Language Model Safety for Polish ACL 2025 Plot2Code: A Comprehensive Benchmark for Evaluating Multi-modal Large Language Models in Code Generation from Scientific Plots NAACL 2025 Polishing Every Facet of the GEM: Testing Linguistic Competence of LLMs and Humans in Korean ACL 2025 Position: Medical Large Language Model Benchmarks Should Prioritize Construct Validity ICML 2025 PoSum-Bench: Benchmarking Position Bias in LLM-based Conversational Summarization EMNLP 2025 Pragmatics in the Era of Large Language Models: A Survey on Datasets, Evaluation, Opportunities and Challenges ACL 2025 PRMBench: A Fine-grained and Challenging Benchmark for Process-Level Reward Models ACL 2025 Probing the Mid-level Vision Capabilities of Self-Supervised Learning CVPR 2025 Problem Solved? Information Extraction Design Space for Layout-Rich Documents using LLMs EMNLP 2025 ProcVQA: Benchmarking the Effects of Structural Properties in Mined Process Visualizations on Vision–Language Model Performance EMNLP 2025 Prompting Large Language Models to Tackle the Full Software Development Lifecycle: A Case Study COLING 2025 ProteinBench: A Holistic Evaluation of Protein Foundation Models ICLR 2025 PSET: a Phonetics-Semantics Evaluation Testbed EMNLP 2025 Pun Unintended: LLMs and the Illusion of Humor Understanding EMNLP 2025 Q-Bench-Video: Benchmark the Video Quality Understanding of LMMs CVPR 2025 QualBench: Benchmarking Chinese LLMs with Localized Professional Qualifications for Vertical Domain Evaluation EMNLP 2025 QUENCH: Measuring the gap between Indic and Non-Indic Contextual General Reasoning in LLMs COLING 2025 Questioning Our Questions: How Well Do Medical QA Benchmarks Evaluate Clinical Capabilities of Language Models? ACL 2025 R2I-Bench: Benchmarking Reasoning-Driven Text-to-Image Generation EMNLP 2025 RAG-RewardBench: Benchmarking Reward Models in Retrieval Augmented Generation for Preference Alignment ACL 2025 RCI: A Score for Evaluating Global and Local Reasoning in Multimodal Benchmarks EMNLP 2025 RealBench: A Chinese Multi-image Understanding Benchmark Close to Real-world Scenarios EMNLP 2025