conftrace_

human judgment

74 papers

Explore in graph

Co-occurring keywords

evaluation metric (573) large language model (13587) machine translation (2499) automatic evaluation (225) language model (4599) human evaluation (379) image captioning (731) text summarization (896) metric correlation (23) automatic metric (65)

Papers

Rethinking Creativity Evaluation: A Critical Analysis of Existing Creativity Evaluations EACL 2026
Revisiting Metric Reliability for Fine-grained Evaluation of Machine Translation and Summarization in Indian Languages ACL 2026
The Progress Illusion: Revisiting meta-evaluation standards of LLM evaluators EMNLP 2025
RealEdit: Reddit Edits As a Large-scale Empirical Dataset for Image Transformations CVPR 2025
Gathering Compositionality Ratings of Ambiguous Noun-Adjective Multiword Expressions in Galician NAACL 2025
Image Captioning Evaluation in the Age of Multimodal LLMs: Challenges and Future Perspectives IJCAI 2025
Readability Reconsidered A Cross-Dataset Analysis of Reference-Free Metrics EMNLP 2025
LLMs instead of Human Judges? A Large Scale Empirical Study across 20 NLP Evaluation Tasks ACL 2025
Aligning Black-box Language Models with Human Judgments NAACL 2025
Debatable Intelligence: Benchmarking LLM Judges via Debate Speech Evaluation EMNLP 2025
From Calculation to Adjudication: Examining LLM Judges on Mathematical Reasoning Tasks ACL 2025
Not ready for the bench: LLM legal interpretation is unstable and uncalibrated to human judgments EMNLP 2025
Cross-Lingual Auto Evaluation for Assessing Multilingual LLMs ACL 2025
VELA: An LLM-Hybrid-as-a-Judge Approach for Evaluating Long Image Captions EMNLP 2025
Long-context Reference-based MT Quality Estimation EMNLP 2025
Maximizing Signal in Human-Model Preference Alignment AAAI 2025
Evaluating the Evaluators: Are readability metrics good measures of readability? EMNLP 2025
SimulatorArena: Are User Simulators Reliable Proxies for Multi-Turn Evaluation of AI Assistants? EMNLP 2025
Measuring the Robustness of Reference-Free Dialogue Evaluation Systems COLING 2025
A Calibrated Reflection Approach for Enhancing Confidence Estimation in LLMs NAACL 2025
Human Bias in the Face of AI: Examining Human Judgment Against Text Labeled as AI Generated ACL 2025
Arbiters of Ambivalence: Challenges of using LLMs in No-Consensus tasks ACL 2025
Beyond Reference: Evaluating High Quality Translations Better than Human References EMNLP 2024
Adjusting Interpretable Dimensions in Embedding Space with Human Judgments NAACL 2024
Prometheus 2: An Open Source Language Model Specialized in Evaluating Other Language Models EMNLP 2024