human judgement

Evaluation of Multilingual Image Captioning: How far can we get with CLIP models? NAACL 2025 Sentence-level Aggregation of Lexical Metrics Correlates Stronger with Human Judgements than Corpus-level Aggregation AAAI 2025 Rethinking STS and NLI in Large Language Models EACL 2024 DiQAD: A Benchmark Dataset for Open-domain Dialogue Quality Assessment EMNLP 2023 Large-Scale Correlation Analysis of Automated Metrics for Topic Models ACL 2023 Rethinking the Word-level Quality Estimation for Machine Translation from Human Judgement ACL 2023 Syntax and Semantics Meet in the “Middle”: Probing the Syntax-Semantics Interface of LMs Through Agentivity ACL 2023 ACES: Translation Accuracy Challenge Sets for Evaluating Machine Translation Metrics EMNLP 2022 MS-COMET: More and Better Human Judgements Improve Metric Performance EMNLP 2022 A Comprehensive Assessment of Dialog Evaluation Metrics EMNLP 2021 Is this the end of the gold standard? A straightforward reference-less grammatical error correction metric EMNLP 2021 Swords: A Benchmark for Lexical Substitution with Improved Data Coverage and Quality NAACL 2021 The Acceptability Delta Criterion: Testing Knowledge of Language using the Gradience of Sentence Acceptability EMNLP 2021 To Ship or Not to Ship: An Extensive Evaluation of Automatic Metrics for Machine Translation EMNLP 2021 COMET: A Neural Framework for MT Evaluation EMNLP 2020 Putting Evaluation in Context: Contextual Embeddings Improve Machine Translation Evaluation ACL 2019 ITER: Improving Translation Edit Rate through Optimizable Edit Costs EMNLP 2018 Image Specificity CVPR 2015