llm evaluation

Ask Me Like I’m Human: LLM-based Evaluation with For-Human Instructions Correlates Better with Human Evaluations than Human Judges ACL 2025 Assessing Crowdsourced Annotations with LLMs: Linguistic Certainty as a Proxy for Trustworthiness NAACL 2025 Assessing the Human Likeness of AI-Generated Counterspeech COLING 2025 Auto-Arena: Automating LLM Evaluations with Agent Peer Battles and Committee Discussions ACL 2025 Automatic Multiple-Choice Question Generation and Evaluation Systems Based on LLM: A Study Case With University Resolutions COLING 2025 Beyond Pointwise Scores: Decomposed Criteria-Based Evaluation of LLM Responses EMNLP 2025 Can External Validation Tools Improve Annotation Quality for LLM-as-a-Judge? ACL 2025 Can LLMs Generate High-Quality Test Cases for Algorithm Problems? TestCase-Eval: A Systematic Evaluation of Fault Coverage and Exposure ACL 2025 Can LLMs Narrate Tabular Data? An Evaluation Framework for Natural Language Representations of Text-to-SQL System Outputs EMNLP 2025 Can LLMs Reason Abstractly Over Math Word Problems Without CoT? Disentangling Abstract Formulation From Arithmetic Computation EMNLP 2025 Can You Trick the Grader? Adversarial Persuasion of LLM Judges EMNLP 2025 CharacterBox: Evaluating the Role-Playing Capabilities of LLMs in Text-Based Virtual Worlds NAACL 2025 Characterizing Positional Bias in Large Language Models: A Multi-Model Evaluation of Prompt Order Effects EMNLP 2025 CheckEval: A reliable LLM-as-a-Judge framework for evaluating text generation using checklists EMNLP 2025 Co-Eval: Augmenting LLM-based Evaluation with Machine Metrics EMNLP 2025 CoreEval: Automatically Building Contamination-Resilient Datasets with Real-World Knowledge toward Reliable LLM Evaluation ACL 2025 Counterfactual Evaluation for Blind Attack Detection in LLM-based Evaluation Systems IJCNLP 2025 Crowd Comparative Reasoning: Unlocking Comprehensive Evaluations for LLM-as-a-Judge ACL 2025 Do LLMs Align Human Values Regarding Social Biases? Judging and Explaining Social Biases with LLMs EMNLP 2025 Do LLMs Behave as Claimed? Investigating How LLMs Follow Their Own Claims using Counterfactual Questions EMNLP 2025 DOVE: A Large-Scale Multi-Dimensional Predictions Dataset Towards Meaningful LLM Evaluation ACL 2025 Drivel-ology: Challenging LLMs with Interpreting Nonsense with Depth EMNLP 2025 EmoAgent: Assessing and Safeguarding Human-AI Interaction for Mental Health Safety EMNLP 2025 Evaluating Personalized Tool-Augmented LLMs from the Perspectives of Personalization and Proactivity ACL 2025 Evaluation Sheet for Deep Research: A Use Case for Academic Survey Writing EMNLP 2025