language model evaluation

Meaning Beyond Truth Conditions: Evaluating Discourse Level Understanding via Anaphora Accessibility ACL 2025 Measuring the Effect of Disfluency in Multilingual Knowledge Probing Benchmarks EMNLP 2025 M-IFEval: Multilingual Instruction-Following Evaluation NAACL 2025 Mind the Style Gap: Meta-Evaluation of Style and Attribute Transfer Metrics EMNLP 2025 neDIOM: Dataset and Analysis of Nepali Idioms COLING 2025 Not quite Sherlock Holmes: Language model predictions do not reliably differentiate impossible from improbable events ACL 2025 OLMES: A Standard for Language Model Evaluations NAACL 2025 On Speeding Up Language Model Evaluation ICLR 2025 Open Ko-LLM Leaderboard2: Bridging Foundational and Practical Evaluation for Korean LLMs NAACL 2025 Optimization before Evaluation: Evaluation with Unoptimized Prompts Can be Misleading ACL 2025 Pattern Recognition or Medical Knowledge? The Problem with Multiple-Choice Questions in Medicine ACL 2025 RAG-Zeval: Enhancing RAG Responses Evaluator through End-to-End Reasoning and Ranking-Based Reinforcement Learning EMNLP 2025 Reasoning or Memorization? Investigating LLMs’ Capability in Restoring Chinese Internet Homophones ACL 2025 Revitalizing Saturated Benchmarks: A Weighted Metric Approach for Differentiating Large Language Model Performance NAACL 2025 RTP-LX: Can LLMs Evaluate Toxicity in Multilingual Scenarios? AAAI 2025 Semantic Inversion, Identical Replies: Revisiting Negation Blindness in Large Language Models EMNLP 2025 skLEP: A Slovak General Language Understanding Benchmark ACL 2025 SocialEval: Evaluating Social Intelligence of Large Language Models ACL 2025 TEXT-CAKE: Challenging Language Models on Local Text Coherence COLING 2025 That Ain’t Right: Assessing LLM Performance on QA in African American and West African English Dialects EMNLP 2025 The BiGGen Bench: A Principled Benchmark for Fine-grained Evaluation of Language Models with Language Models NAACL 2025 Think Globally, Group Locally: Evaluating LLMs Using Multi-Lingual Word Grouping Games EMNLP 2025 TounsiBench: Benchmarking Large Language Models for Tunisian Arabic EMNLP 2025 Trick or Neat: Adversarial Ambiguity and Language Model Evaluation ACL 2025 UNCLE: Benchmarking Uncertainty Expressions in Long-Form Generation EMNLP 2025