human judgment

Rethinking Creativity Evaluation: A Critical Analysis of Existing Creativity Evaluations EACL 2026 Revisiting Metric Reliability for Fine-grained Evaluation of Machine Translation and Summarization in Indian Languages ACL 2026 Sense and Sensitivity: “Reasoning” Models are More Robust, but can Diverge from Human Consensus in a Legal Interpretation Task CoNLL 2026 A Calibrated Reflection Approach for Enhancing Confidence Estimation in LLMs NAACL 2025 Aligning Black-box Language Models with Human Judgments NAACL 2025 Arbiters of Ambivalence: Challenges of using LLMs in No-Consensus tasks ACL 2025 A Statistical Framework for Ranking LLM-based Chatbots ICLR 2025 Cross-Lingual Auto Evaluation for Assessing Multilingual LLMs ACL 2025 Debatable Intelligence: Benchmarking LLM Judges via Debate Speech Evaluation EMNLP 2025 Evaluating the Evaluators: Are readability metrics good measures of readability? EMNLP 2025 From Calculation to Adjudication: Examining LLM Judges on Mathematical Reasoning Tasks ACL 2025 Gathering Compositionality Ratings of Ambiguous Noun-Adjective Multiword Expressions in Galician NAACL 2025 Human Bias in the Face of AI: Examining Human Judgment Against Text Labeled as AI Generated ACL 2025 Image Captioning Evaluation in the Age of Multimodal LLMs: Challenges and Future Perspectives IJCAI 2025 LLMs instead of Human Judges? A Large Scale Empirical Study across 20 NLP Evaluation Tasks ACL 2025 Long-context Reference-based MT Quality Estimation EMNLP 2025 Maximizing Signal in Human-Model Preference Alignment AAAI 2025 Measuring the Robustness of Reference-Free Dialogue Evaluation Systems COLING 2025 Not ready for the bench: LLM legal interpretation is unstable and uncalibrated to human judgments EMNLP 2025 Readability Reconsidered A Cross-Dataset Analysis of Reference-Free Metrics EMNLP 2025 RealEdit: Reddit Edits As a Large-scale Empirical Dataset for Image Transformations CVPR 2025 SimulatorArena: Are User Simulators Reliable Proxies for Multi-Turn Evaluation of AI Assistants? EMNLP 2025 The Progress Illusion: Revisiting meta-evaluation standards of LLM evaluators EMNLP 2025 VELA: An LLM-Hybrid-as-a-Judge Approach for Evaluating Long Image Captions EMNLP 2025 Adjusting Interpretable Dimensions in Embedding Space with Human Judgments NAACL 2024