human judgment

Pre-trained language models evaluating themselves - A comparative study ACL 2022 Re-Examining System-Level Correlations of Automatic Summarization Evaluation Metrics NAACL 2022 Revisiting Automatic Evaluation of Extractive Summarization Task: Can We Do Better than ROUGE? ACL 2022 On the Stability of System Rankings at WMT EMNLP 2021 QuestEval: Summarization Asks for Fact-based Evaluation EMNLP 2021 RAW-C: Relatedness of Ambiguous Words in Context (A New Lexical Resource for English) ACL 2021 Semantic Similarity Based Evaluation for Abstractive News Summarization ACL 2021 To Ship or Not to Ship: An Extensive Evaluation of Automatic Metrics for Machine Translation EMNLP 2021 HOLMS: Alternative Summary Evaluation with Large Language Models COLING 2020 Human-Paraphrased References Improve Neural Machine Translation EMNLP 2020 SacreROUGE: An Open-Source Library for Using and Developing Summarization Evaluation Metrics EMNLP 2020 Towards a Better Evaluation of Metrics for Machine Translation EMNLP 2020 UNION: An Unreferenced Metric for Evaluating Open-ended Story Generation EMNLP 2020 Findings of the First Shared Task on Machine Translation Robustness ACL 2019 MoverScore: Text Generation Evaluating with Contextualized Embeddings and Earth Mover Distance EMNLP 2019 REO-Relevance, Extraness, Omission: A Fine-grained Evaluation for Image Captioning EMNLP 2019 Results of the WMT19 Metrics Shared Task: Segment-Level and Strong MT Systems Pose Big Challenges ACL 2019 Studying Summarization Evaluation Metrics in the Appropriate Scoring Range ACL 2019 BLEU is Not Suitable for the Evaluation of Text Simplification EMNLP 2018 Findings of the 2018 Conference on Machine Translation (WMT18) EMNLP 2018 Objective Function Learning to Match Human Judgements for Optimization-Based Summarization NAACL 2018 A Principled Framework for Evaluating Summarizers: Comparing Models of Summary Quality against Human Judgments ACL 2017 Towards an Automatic Turing Test: Learning to Evaluate Dialogue Responses ACL 2017 An Unsupervised Decontamination Procedure For Improving The Reliability Of Human Judgments NeurIPS 2011 A rational model of causal inference with continuous causes NeurIPS 2011