automatic metric

Revisiting Metric Reliability for Fine-grained Evaluation of Machine Translation and Summarization in Indian Languages ACL 2026 EditBoard: Towards a Comprehensive Evaluation Benchmark for Text-Based Video Editing Models AAAI 2025 Evaluation of LLM for English to Hindi Legal Domain Machine Translation Systems EMNLP 2025 Has Machine Translation Evaluation Achieved Human Parity? The Human Reference and the Limits of Progress ACL 2025 How Good Are LLMs for Literary Translation, Really? Literary Translation Evaluation with Humans and LLMs NAACL 2025 LiteraryQA: Towards Effective Evaluation of Long-document Narrative QA EMNLP 2025 MSLC25: Metric Performance on Low-Quality Machine Translation, Empty Strings, and Language Variants EMNLP 2025 UTER: Capturing the Human Touch in Evaluating Morphologically Rich and Low-Resource Languages NAACL 2025 WMT24++: Expanding the Language Coverage of WMT24 to 55 Languages & Dialects ACL 2025 Are LLMs Breaking MT Metrics? Results of the WMT24 Metrics Shared Task EMNLP 2024 Can Automatic Metrics Assess High-Quality Translations? EMNLP 2024 ChronoMagic-Bench: A Benchmark for Metamorphic Evaluation of Text-to-Time-lapse Video Generation NeurIPS 2024 Evaluating Automatic Metrics with Incremental Machine Translation Systems EMNLP 2024 Evaluating the IWSLT2023 Speech Translation Tasks: Human Annotations, Automatic Metrics, and Segmentation COLING 2024 Findings of WMT 2024 Shared Task on Low-Resource Indic Languages Translation EMNLP 2024 Findings of WMT 2024’s MultiIndic22MT Shared Task for Machine Translation of 22 Indian Languages EMNLP 2024 How Much Annotation is Needed to Compare Summarization Models? NAACL 2024 Improving Statistical Significance in Human Evaluation of Automatic Metrics via Soft Pairwise Accuracy EMNLP 2024 InfiBench: Evaluating the Question-Answering Capabilities of Code Large Language Models NeurIPS 2024 MetricX-24: The Google Submission to the WMT 2024 Metrics Shared Task EMNLP 2024 Modeling User Preferences with Automatic Metrics: Creating a High-Quality Preference Dataset for Machine Translation EMNLP 2024 MSLC24: Further Challenges for Metrics on a Wide Landscape of Translation Quality EMNLP 2024 QGEval: Benchmarking Multi-dimensional Evaluation for Question Generation EMNLP 2024 Quality and Quantity of Machine Translation References for Automatic Metrics COLING 2024 VideoScore: Building Automatic Metrics to Simulate Fine-grained Human Feedback for Video Generation EMNLP 2024