benchmark evaluation

Lived Experience Not Found: LLMs Struggle to Align with Experts on Addressing Adverse Drug Reactions from Psychiatric Medication Use NAACL 2025 LiveXiv - A Multi-Modal live benchmark based on Arxiv papers content ICLR 2025 LLäMmlein: Transparent, Compact and Competitive German-Only Language Models from Scratch ACL 2025 LLM Agents Making Agent Tools ACL 2025 LLM-Coordination: Evaluating and Analyzing Multi-agent Coordination Abilities in Large Language Models NAACL 2025 LLMs Are Not Intelligent Thinkers: Introducing Mathematical Topic Tree Benchmark for Comprehensive Evaluation of LLMs NAACL 2025 LLMs can be easily Confused by Instructional Distractions ACL 2025 LLMs instead of Human Judges? A Large Scale Empirical Study across 20 NLP Evaluation Tasks ACL 2025 LLMsPark: A Benchmark for Evaluating Large Language Models in Strategic Gaming Contexts EMNLP 2025 LMR-BENCH: Evaluating LLM Agent’s Ability on Reproducing Language Modeling Research EMNLP 2025 LNE-Blocking: An Efficient Framework for Contamination Mitigation Evaluation on Large Language Models EMNLP 2025 LogicGame: Benchmarking Rule-Based Reasoning Abilities of Large Language Models ACL 2025 LongBench v2: Towards Deeper Understanding and Reasoning on Realistic Long-context Multitasks ACL 2025 LongDocURL: a Comprehensive Multimodal Long Document Benchmark Integrating Understanding, Reasoning, and Locating ACL 2025 LongSafety: Enhance Safety for Long-Context LLMs ACL 2025 LongSafety: Evaluating Long-Context Safety of Large Language Models ACL 2025 LongTableBench: Benchmarking Long-Context Table Reasoning across Real-World Formats and Domains EMNLP 2025 LongWeave: A Long-Form Generation Benchmark Bridging Real-World Relevance and Verifiability EMNLP 2025 LR²Bench: Evaluating Long-chain Reflective Reasoning Capabilities of Large Language Models via Constraint Satisfaction Problems ACL 2025 M2RC-EVAL: Massively Multilingual Repository-level Code Completion Evaluation ACL 2025 M3Retrieve: Benchmarking Multimodal Retrieval for Medicine EMNLP 2025 Machine Unlearning of Personally Identifiable Information in Large Language Models EMNLP 2025 MADial-Bench: Towards Real-world Evaluation of Memory-Augmented Dialogue Generation NAACL 2025 MANBench: Is Your Multimodal Model Smarter than Human? ACL 2025 MARS: Benchmarking the Metaphysical Reasoning Abilities of Language Models with a Multi-task Evaluation Dataset ACL 2025