summarization evaluation

84 papers

Explore in graph

Co-occurring keywords

human evaluation (375) large language model (12755) text summarization (889) reference-free evaluation (88) evaluation metric (415) natural language inference (1278) rouge metric (25) text generation (2903) abstractive summarization (631) automatic evaluation (224)

Papers

RISE: Leveraging Retrieval Techniques for Summarization Evaluation ACL 2023

TrueTeacher: Learning Factual Consistency Evaluation with Large Language Models EMNLP 2023

Open Domain Multi-document Summarization: A Comprehensive Study of Model Brittleness under Retrieval EMNLP 2023

Entity-Based Evaluation of Political Bias in Automatic Summarization EMNLP 2023

LTRC_IIITH’s 2023 Submission for Prompting Large Language Models as Explainable Metrics Task AACL 2023

Characterised LLMs Affect its Evaluation of Summary and Translation AACL 2023

Reference-Free Summarization Evaluation with Large Language Models AACL 2023

Towards Better Evaluation of Instruction-Following: A Case-Study in Summarization CONLL 2023

LongEval: Guidelines for Human Evaluation of Faithfulness in Long-form Summarization EACL 2023

Understanding Large Language Model Based Metrics for Text Summarization IJCNLP 2023

SEAHORSE: A Multilingual, Multifaceted Dataset for Summarization Evaluation EMNLP 2023

Extractive is not Faithful: An Investigation of Broad Unfaithfulness Problems in Extractive Summarization ACL 2023

Revisiting the Gold Standard: Grounding Summarization Evaluation with Robust Human Evaluation ACL 2023

SummEdits: Measuring LLM Ability at Factual Reasoning Through The Lens of Summarization EMNLP 2023

A Study on Annotation Interfaces for Summary Comparison ACL 2023

The Eval4NLP 2023 Shared Task on Prompting Large Language Models as Explainable Metrics IJCNLP 2023

SEM-F1: an Automatic Way for Semantic Evaluation of Multi-Narrative Overlap Summaries at Scale EMNLP 2022

InfoLM: A New Metric to Evaluate Summarization & Data2Text Generation AAAI 2022

Revisiting text decomposition methods for NLI-based factuality scoring of summaries EMNLP 2022

Questioning the Validity of Summarization Datasets and Improving Their Factual Consistency EMNLP 2022

PrefScore: Pairwise Preference Learning for Reference-free Summarization Quality Assessment COLING 2022

SueNes: A Weakly Supervised Approach to Evaluating Single-Document Summarization via Negative Sampling NAACL 2022

Investigating Metric Diversity for Evaluating Long Document Summarisation COLING 2022

DialSummEval: Revisiting Summarization Evaluation for Dialogues NAACL 2022

Investigating Crowdsourcing Protocols for Evaluating the Factual Consistency of Summaries NAACL 2022