summarization evaluation

Auto-PRE: An Automatic and Cost-Efficient Peer-Review Framework for Language Generation Evaluation AAAI 2026 A Tale of Evaluating Factual Consistency: Case Study on Long Document Summarization Evaluation ACL 2025 ContrastScore: Towards Higher Quality, Less Biased, More Efficient Evaluation Metrics with Contrastive Evaluation IJCNLP 2025 Real-World Summarization: When Evaluation Reaches Its Limits EMNLP 2025 ReproHum #0067-01: A Reproduction of the Evaluation of Cross-Lingual Summarization ACL 2025 ReproHum #0729-04: Human Evaluation Reproduction Report for “MemSum: Extractive Summarization of Long Documents Using Multi-Step Episodic Markov Decision Processes” ACL 2025 Spot the BlindSpots: Systematic Identification and Quantification of Fine-Grained LLM Biases in Contact Center Call Summarization EMNLP 2025 Who Remembers What? Tracing Information Fidelity in Human-AI Chains IJCNLP 2025 A Critical Look at Meta-evaluating Summarisation Evaluation Metrics EMNLP 2024 COSMIC: Mutual Information for Task-Agnostic Summarization Evaluation ACL 2024 METAL: Towards Multilingual Meta-Evaluation NAACL 2024 Mitigating the Impact of Reference Quality on Evaluation of Summarization Systems with Reference-Free Metrics EMNLP 2024 One Prompt To Rule Them All: LLMs for Opinion Summary Evaluation ACL 2024 PrExMe! Large Scale Prompt Exploration of Open Source LLMs for Machine Translation and Summarization Evaluation EMNLP 2024 ROUGE-K: Do Your Summaries Have Keywords? NAACL 2024 Summary of a Haystack: A Challenge to Long-Context LLMs and RAG Systems EMNLP 2024 SummEQuAL: Summarization Evaluation via Question Answering using Large Language Models ACL 2024 UniSumEval: Towards Unified, Fine-grained, Multi-dimensional Summarization Evaluation for LLMs EMNLP 2024 A Needle in a Haystack: An Analysis of High-Agreement Workers on MTurk for Summarization ACL 2023 A Study on Annotation Interfaces for Summary Comparison ACL 2023 BUMP: A Benchmark of Unfaithful Minimal Pairs for Meta-Evaluation of Faithfulness Metrics ACL 2023 Characterised LLMs Affect its Evaluation of Summary and Translation AACL 2023 Entity-Based Evaluation of Political Bias in Automatic Summarization EMNLP 2023 Extractive is not Faithful: An Investigation of Broad Unfaithfulness Problems in Extractive Summarization ACL 2023 LongEval: Guidelines for Human Evaluation of Faithfulness in Long-form Summarization EACL 2023