conftrace_

annotation reliability

6 papers

Explore in graph

Co-occurring keywords · papers mentioning both

inter-annotator agreement (2) annotation quality (2) sentiment annotation (1) best-worst scaling (1) rating scale (1) annotator consistency (1) sentiment intensity (1) annotation disagreement (1) linguistic meaning (1) prague dependency treebank (1)

Papers

“All that Glitters”: Techniques for Evaluations with Unreliable Model and Human Annotations NAACL 2025 Label Bias in Symbolic Representation of Meaning ACL 2025 A Gold Standard with Silver Linings: Scaling Up Annotation for Distinguishing Bosnian, Croatian, Montenegrin and Serbian COLING 2024 Estimating Agreement by Chance for Sequence Annotation ACL 2024 Identifying Annotator Bias: A new IRT-based method for bias identification COLING 2020 Best-Worst Scaling More Reliable than Rating Scales: A Case Study on Sentiment Intensity Annotation ACL 2017