system evaluation

Evaluation and Incident Prevention in an Enterprise AI Assistant AAAI 2025 Investigating the Robustness of Retrieval-Augmented Generation at the Query Level ACL 2025 ReproHum #0729-04: Partial reproduction of the human evaluation of the MemSum and NeuSum summarisation systems ACL 2025 SPHERE: An Evaluation Card for Human-AI Systems ACL 2025 CoST of breaking the LLMs EMNLP 2024 Fusion-Eval: Integrating Assistant Evaluators with LLMs EMNLP 2024 Towards More Robust NLP System Evaluation: Handling Missing Scores in Benchmarks EMNLP 2024 Unveiling Strengths and Weaknesses of NLP Systems Based on a Rich Evaluation Corpus: The Case of NER in French COLING 2024 Mass-Producing Failures of Multimodal Systems with Language Models NeurIPS 2023 PriMeSRL-Eval: A Practical Quality Metric for Semantic Role Labeling Systems Evaluation EACL 2023 Reduce Human Labor On Evaluating Conversational Information Retrieval System: A Human-Machine Collaboration Approach EMNLP 2023 Surrogate Assisted Generation of Human-Robot Interaction Scenarios CoRL 2023 Vote’n’Rank: Revision of Benchmarking with Social Choice Theory EACL 2023 Part Represents Whole: Improving the Evaluation of Machine Translation System Using Entropy Enhanced Metrics AACL 2022 Quantified Reproducibility Assessment of NLP Results ACL 2022 The Dangers of Underclaiming: Reasons for Caution When Reporting How NLP Systems Fail ACL 2022 Towards Robust Numerical Question Answering: Diagnosing Numerical Capabilities of NLP Systems EMNLP 2022 Anaphora Resolution in Dialogue: Cross-Team Analysis of the DFKI-TalkingRobots Team Submissions for the CODI-CRAC 2021 Shared-Task EMNLP 2021 Better than Average: Paired Evaluation of NLP systems ACL 2021 Differential Evaluation: a Qualitative Analysis of Natural Language Processing System Behavior Based Upon Data Resistance to Processing EMNLP 2021 Evaluation of Summarization Systems across Gender, Age, and Race EMNLP 2021 How Good (really) are Grammatical Error Correction Systems? EACL 2021 To Ship or Not to Ship: An Extensive Evaluation of Automatic Metrics for Machine Translation EMNLP 2021 What is SemEval evaluating? A Systematic Analysis of Evaluation Campaigns in NLP EMNLP 2021 How Do Image Description Systems Describe People? A Targeted Assessment of System Competence in the PEOPLE-domain COLING 2020