annotator agreement

On Crowdsourcing Task Design for Discourse Relation Annotation COLING 2025 PERSEVAL: A Framework for Perspectivist Classification Evaluation EMNLP 2025 SemEval-2025 Task 3: Mu-SHROOM, the Multilingual Shared-task on Hallucinations and Related Observable Overgeneration Mistakes SemEval 2025 Sources of Disagreement in Data for LLM Instruction Tuning COLING 2025 Can You Rely on Synthetic Labellers in Preference-Based Reinforcement Learning? It’s Complicated AAAI 2024 Collaborative Refining for Learning from Inaccurate Labels NeurIPS 2024 Deloitte at #SMM4H 2024: Can GPT-4 Detect COVID-19 Tweets Annotated by Itself? ACL 2024 Do PLMs and Annotators Share the Same Gender Bias? Definition, Dataset, and Framework of Contextualized Gender Bias ACL 2024 Multi-Reference Benchmarks for Russian Grammatical Error Correction EACL 2024 Annotators-in-the-loop: Testing a Novel Annotation Procedure on Italian Case Law ACL 2023 Temporal and Second Language Influence on Intra-Annotator Agreement and Stability in Hate Speech Labelling ACL 2023 Adversarially Constructed Evaluation Sets Are More Challenging, but May Not Be Fair NAACL 2022 SenTag: A Web-Based Tool for Semantic Annotation of Textual Documents AAAI 2022 The Authenticity Gap in Human Evaluation EMNLP 2022 The Effect of Arabic Dialect Familiarity on Data Annotation EMNLP 2022 Learn Like a Pathologist: Curriculum Learning by Annotator Agreement for Histopathology Image Classification WACV 2021 What Sounds “Right” to Me? Experiential Factors in the Perception of Political Ideology EACL 2021 Crossing the Line: Where do Demographic Variables Fit into Humor Detection? ACL 2020 Is this Sentence Difficult? Do you Agree? EMNLP 2018