inter-annotator agreement

ReproHum: #0744-02: Investigating the Reproducibility of Semantic Preservation Human Evaluations ACL 2025 Subjectivity in the Annotation of Bridging Anaphora ACL 2025 T5Score: A Methodology for Automatically Assessing the Quality of LLM Generated Multi-Document Topic Sets ACL 2025 The Alternative Annotator Test for LLM-as-a-Judge: How to Statistically Justify Replacing Human Annotators with LLMs ACL 2025 The Incremental Process of Building an Annotation Scheme for Clinical Narratives in Portuguese: the Contribution of Human Variation Analysis ACL 2025 UniT: One Document, Many Revisions, Too Many Edit Intention Taxonomies ACL 2025 Where it’s at: Annotating Verb Placement Types in Learner Language ACL 2025 A Corpus of German Abstract Meaning Representation (DeAMR) COLING 2024 A General Model for Aggregating Annotations AcrossSimple, Complex, and Multi-object Annotation Tasks (Abstract Reprint) AAAI 2024 A GPT among Annotators: LLM-based Entity-Level Sentiment Annotation EACL 2024 Annotating Chinese Word Senses with English WordNet: A Practice on OntoNotes Chinese Sense Inventories COLING 2024 Are You Serious? Handling Disagreement When Annotating Conspiracy Theory Texts EACL 2024 Bias Bluff Busters at FIGNEWS 2024 Shared Task: Developing Guidelines to Make Bias Conscious ACL 2024 Can Humans Identify Domains? COLING 2024 Can You Rely on Synthetic Labellers in Preference-Based Reinforcement Learning? It’s Complicated AAAI 2024 Different Tastes of Entities: Investigating Human Label Variation in Named Entity Annotations EACL 2024 Disagreement in Argumentation Annotation COLING 2024 Estimating Agreement by Chance for Sequence Annotation ACL 2024 Estimating the Level of Dialectness Predicts Inter-annotator Agreement in Multi-dialect Arabic Datasets ACL 2024 Evaluating the Development of Linguistic Metaphor Annotation in Mexican Spanish Popular Science Tweets NAACL 2024 GroningenAnnotatesGaza at the FIGNEWS 2024 Shared Task: Analyzing Bias in Conflict Narratives ACL 2024 How to Annotate Emotions in Historical Italian Novels: A Case Study on I Promessi Sposi COLING 2024 LeadEmpathy: An Expert Annotated German Dataset of Empathy in Written Leadership Communication COLING 2024 Multi-Dimensional Insights: Annotated Dataset of Stance, Sentiment, and Emotion in Facebook Comments on Tunisia’s July 25 Measures COLING 2024 Narrative Navigators at FIGNEWS 2024 Shared Task: New Frontiers in Bias and Propaganda Annotation Techniques ACL 2024