inter-annotator agreement

A Novel Annotation Schema for Conversational Humor: Capturing the Cultural Nuances in Kanyasulkam COLING 2020 Effective Crowd-Annotation of Participants, Interventions, and Outcomes in the Text of Clinical Trial Reports EMNLP 2020 Identifying Annotator Bias: A new IRT-based method for bias identification COLING 2020 On the Same Page? Comparing Inter-Annotator Agreement in Sentence and Document Level Human Machine Translation Evaluation EMNLP 2020 Uncertainty over Uncertainty: Investigating the Assumptions, Annotations, and Text Measurements of Economic Policy Uncertainty EMNLP 2020 Would you describe a leopard as yellow? Evaluating crowd-annotations with justified and informative disagreement COLING 2020 Annotating and analyzing the interactions between meaning relations ACL 2019 Crowdsourced Hedge Term Disambiguation ACL 2019 Crowdsourcing Discourse Relation Annotations by a Two-Step Connective Insertion Task ACL 2019 Explaining Simple Natural Language Inference ACL 2019 HighRES: Highlight-based Reference-less Evaluation of Summarization ACL 2019 Preparing SNACS for Subjects and Objects ACL 2019 RST-Tace A tool for automatic comparison and evaluation of RST trees NAACL 2019 Towards an Annotation Scheme for Complex Laughter in Speech Corpora Interspeech 2019 A Laypeople Study on Terminology Identification across Domains and Task Definitions NAACL 2018 A Multi-Axis Annotation Scheme for Event Temporal Relations ACL 2018 An Annotated Corpus of Picture Stories Retold by Language Learners COLING 2018 Annotating Claims in the Vaccination Debate EMNLP 2018 Annotating picture description task responses for content analysis NAACL 2018 Attaining the Unattainable? Reassessing Claims of Human Parity in Neural Machine Translation EMNLP 2018 Constructing an Annotated Corpus of Verbal MWEs for English COLING 2018 Feasible Annotation Scheme for Capturing Policy Argument Reasoning using Argument Templates EMNLP 2018 Is this Sentence Difficult? Do you Agree? EMNLP 2018 Iterative development of family history annotation guidelines using a synthetic corpus of clinical text EMNLP 2018 Rethinking the Agreement in Human Evaluation Tasks COLING 2018