conftrace_

human judgment

74 papers

Explore in graph

Co-occurring keywords

evaluation metric (573) large language model (13587) machine translation (2499) automatic evaluation (225) language model (4599) human evaluation (379) image captioning (731) text summarization (896) metric correlation (23) automatic metric (65)

Papers

On the Stability of System Rankings at WMT EMNLP 2021
Semantic Similarity Based Evaluation for Abstractive News Summarization ACL 2021
To Ship or Not to Ship: An Extensive Evaluation of Automatic Metrics for Machine Translation EMNLP 2021
QuestEval: Summarization Asks for Fact-based Evaluation EMNLP 2021
RAW-C: Relatedness of Ambiguous Words in Context (A New Lexical Resource for English) ACL 2021
Towards a Better Evaluation of Metrics for Machine Translation EMNLP 2020
HOLMS: Alternative Summary Evaluation with Large Language Models COLING 2020
UNION: An Unreferenced Metric for Evaluating Open-ended Story Generation EMNLP 2020
SacreROUGE: An Open-Source Library for Using and Developing Summarization Evaluation Metrics EMNLP 2020
Human-Paraphrased References Improve Neural Machine Translation EMNLP 2020
MoverScore: Text Generation Evaluating with Contextualized Embeddings and Earth Mover Distance EMNLP 2019
REO-Relevance, Extraness, Omission: A Fine-grained Evaluation for Image Captioning EMNLP 2019
Results of the WMT19 Metrics Shared Task: Segment-Level and Strong MT Systems Pose Big Challenges ACL 2019
REO-Relevance, Extraness, Omission: A Fine-grained Evaluation for Image Captioning IJCNLP 2019
Studying Summarization Evaluation Metrics in the Appropriate Scoring Range ACL 2019
Findings of the First Shared Task on Machine Translation Robustness ACL 2019
Objective Function Learning to Match Human Judgements for Optimization-Based Summarization NAACL 2018
BLEU is Not Suitable for the Evaluation of Text Simplification EMNLP 2018
Findings of the 2018 Conference on Machine Translation (WMT18) EMNLP 2018
A Principled Framework for Evaluating Summarizers: Comparing Models of Summary Quality against Human Judgments ACL 2017
Towards an Automatic Turing Test: Learning to Evaluate Dialogue Responses ACL 2017
Testing a Bayesian Measure of Representativeness Using a Large Image Database NIPS 2011
An Unsupervised Decontamination Procedure For Improving The Reliability Of Human Judgments NIPS 2011
Improving Human Judgments by Decontaminating Sequential Dependencies NIPS 2010