llm evaluation

Better Datasets Start from RefineLab: Automatic Optimization for High-Quality Dataset Refinement AAAI 2026 CLEAR: Error Analysis via LLM-as-a-Judge Made Easy AAAI 2026 ConInstruct: Evaluating Large Language Models on Conflict Detection and Resolution in Instructions AAAI 2026 Controllable Contamination Detection for Reliable LLM Evaluation with Statistical Guarantees ACL 2026 Debate, Deliberate, Decide (D3): A Cost-Aware Adversarial Framework for Reliable and Interpretable LLM Evaluation EACL 2026 Democratizing Diplomacy: A Harness for Evaluating Any Large Language Model on Full-Press Diplomacy AAAI 2026 FFE-Hallu: Hallucinations in Fixed Figurative Expressions: A Benchmark of Idioms and Proverbs in the Persian Language EACL 2026 FinMathBench: A Formula-Driven Benchmark for Evaluating LLMs’ Math Reasoning Capabilities in Finance AAAI 2026 Language Models Don’t Know What You Want: Evaluating Personalization in Deep Research Needs Real Users ACL 2026 LLM-as-a-Judge for Low-Resource Languages: Adapting Ragas and Comparative Ranking for Romanian EACL 2026 NarraBench: A Comprehensive Framework for Narrative Benchmarking EACL 2026 Persistent Instability in LLM’s Personality Measurements: Effects of Scale, Reasoning, and Conversation History AAAI 2026 RPC-Bench: A Fine-grained Benchmark for Research Paper Comprehension ACL 2026 The Confidence Trap: Gender Bias and Predictive Certainty in LLMs AAAI 2026 Tracking the evolution of LLM capabilities for Belarusian with OpenAI Evals EACL 2026 Where Norms and References Collide: Evaluating LLMs on Normative Reasoning AAAI 2026 ACEBench: A Comprehensive Evaluation of LLM Tool Usage EMNLP 2025 AdvERSEM: Adversarial Robustness Testing and Training of LLM-based Groundedness Evaluators via Semantic Structure Manipulation EMNLP 2025 AIRepr: An Analyst-Inspector Framework for Evaluating Reproducibility of LLMs in Data Science EMNLP 2025 Algorithmic Fidelity of Large Language Models in Generating Synthetic German Public Opinions: A Case Study ACL 2025 AL-QASIDA: Analyzing LLM Quality and Accuracy Systematically in Dialectal Arabic ACL 2025 An Empirical Study of LLM-as-a-Judge for LLM Evaluation: Fine-tuned Judge Model is not a General Substitute for GPT-4 ACL 2025 AntiLeakBench: Preventing Data Contamination by Automatically Constructing Benchmarks with Updated Real-World Knowledge ACL 2025 AraEval: An Arabic Multi-Task Evaluation Suite for Large Language Models EMNLP 2025 Are LLM-Judges Robust to Expressions of Uncertainty? Investigating the effect of Epistemic Markers on LLM-based Evaluation NAACL 2025