← Keywords

summarization evaluation

84 papers

Explore in graph

Co-occurring keywords

human evaluation (375) large language model (12755) text summarization (889) reference-free evaluation (88) evaluation metric (415) natural language inference (1278) rouge metric (25) text generation (2903) abstractive summarization (631) automatic evaluation (224)

Papers

Does Summary Evaluation Survive Translation to Other Languages? NAACL 2022
SNaC: Coherence Error Detection for Narrative Summarization EMNLP 2022
Reference-free Summarization Evaluation via Semantic Correlation and Compression Ratio NAACL 2022
QAFactEval: Improved QA-Based Factual Consistency Evaluation for Summarization NAACL 2022
Benchmarking Answer Verification Methods for Question Answering-Based Summarization Evaluation Metrics ACL 2022
FALTE: A Toolkit for Fine-grained Annotation for Long Text Evaluation EMNLP 2022
HaRiM+: Evaluating Summary Quality with Hallucination Risk IJCNLP 2022
QuestEval: Summarization Asks for Fact-based Evaluation EMNLP 2021
Understanding the Extent to which Content Quality Metrics Measure the Information Quality of Summaries EMNLP 2021
How to Evaluate a Summarizer: Study Design and Statistical Analysis for Manual Linguistic Quality Evaluation EACL 2021
A Training-free and Reference-free Summarization Evaluation Metric via Centrality-weighted Relevance and Self-referenced Redundancy IJCNLP 2021
Automatic Text Evaluation through the Lens of Wasserstein Barycenters EMNLP 2021
ESTIME: Estimation of Summary-to-Text Inconsistency by Mismatched Embeddings EMNLP 2021
A Training-free and Reference-free Summarization Evaluation Metric via Centrality-weighted Relevance and Self-referenced Redundancy ACL 2021
Fine-grained Factual Consistency Assessment for Abstractive Summarization Models EMNLP 2021
Finding a Balanced Degree of Automation for Summary Evaluation EMNLP 2021
Evaluation of Summarization Systems across Gender, Age, and Race EMNLP 2021
Best Practices for Crowd-based Evaluation of German Summarization: Comparing Crowd, Expert and Automatic Evaluation EMNLP 2020
Truth or Error? Towards systematic analysis of factual errors in abstractive summaries EMNLP 2020
Metrics also Disagree in the Low Scoring Range: Revisiting Summarization Evaluation Metrics COLING 2020
HOLMS: Alternative Summary Evaluation with Large Language Models COLING 2020
SacreROUGE: An Open-Source Library for Using and Developing Summarization Evaluation Metrics EMNLP 2020
Unsupervised Reference-Free Summary Quality Evaluation via Contrastive Learning EMNLP 2020
Ranking Generated Summaries by Correctness: An Interesting but Challenging Application for Natural Language Inference ACL 2019
The Feasibility of Embedding Based Automatic Evaluation for Single Document Summarization EMNLP 2019