annotation analysis

Annotating the Annotators: Analysis, Insights and Modelling from an Annotation Campaign on Persuasion Techniques Detection ACL 2025 Benchmarking the Benchmarks: Reproducing Climate-Related NLP Tasks ACL 2025 Beyond Dataset Creation: Critical View of Annotation Variation and Bias Probing of a Dataset for Online Radical Content Detection COLING 2025 Non-directive corpus annotation to reveal individual perspectives with underspecified guidelines: the case of mental workload EMNLP 2025 Towards a Perspectivist Turn in Argument Quality Assessment NAACL 2025 A Tool for Determining Distances and Overlaps between Multimodal Annotations COLING 2024 Different Tastes of Entities: Investigating Human Label Variation in Named Entity Annotations EACL 2024 More Labels or Cases? Assessing Label Variation in Natural Language Inference EACL 2024 PyRater: A Python Toolkit for Annotation Analysis COLING 2024 Ecologically Valid Explanations for Label Variation in NLI EMNLP 2023 EPIC: Multi-Perspective Annotation of a Corpus of Irony ACL 2023 Evaluating Zero-Shot Event Structures: Recommendations for Automatic Content Extraction (ACE) Annotations ACL 2023 On Event Individuation for Document-Level Information Extraction EMNLP 2023 Why Don’t You Do It Right? Analysing Annotators’ Disagreement in Subjective Tasks EACL 2023 Quantifying Contextual Aspects of Inter-annotator Agreement in Intertextuality Research EMNLP 2021 Uncertainty over Uncertainty: Investigating the Assumptions, Annotations, and Text Measurements of Economic Policy Uncertainty EMNLP 2020 Datasets of Slovene and Croatian Moderated News Comments EMNLP 2018 Latent Confusion Analysis by Normalized Gamma Construction ICML 2014