annotation quality

Strong Empowered and Aligned Weak Mastered Annotation for Weak-to-Strong Generalization AAAI 2025 Task Configuration Impacts Annotation Quality and Model Training Performance in Crowdsourced Image Segmentation WACV 2025 The Alternative Annotator Test for LLM-as-a-Judge: How to Statistically Justify Replacing Human Annotators with LLMs ACL 2025 Where it’s at: Annotating Verb Placement Types in Learner Language ACL 2025 A Gold Standard with Silver Linings: Scaling Up Annotation for Distinguishing Bosnian, Croatian, Montenegrin and Serbian COLING 2024 Automatic Detection and Labelling of Personal Data in Case Reports from the ECHR in Spanish: Evaluation of Two Different Annotation Approaches EACL 2024 Beyond Human-Only: Evaluating Human-Machine Collaboration for Collecting High-Quality Translation Data EMNLP 2024 Context Does Matter: Implications for Crowdsourced Evaluation Labels in Task-Oriented Dialogue Systems NAACL 2024 Efficient Online Crowdsourcing with Complex Annotations AAAI 2024 Estimating the Level of Dialectness Predicts Inter-annotator Agreement in Multi-dialect Arabic Datasets ACL 2024 Finding Replicable Human Evaluations via Stable Ranking Probability NAACL 2024 Geo-Cultural Representation and Inclusion in Language Technologies COLING 2024 “Get Their Hands Dirty, Not Mine”: On Researcher-Annotator Collaboration and the Agency of Annotators ACL 2024 Narrative Navigators at FIGNEWS 2024 Shared Task: New Frontiers in Bias and Propaganda Annotation Techniques ACL 2024 On Efficient and Statistical Quality Estimation for Data Annotation ACL 2024 Order Effects in Annotation Tasks: Further Evidence of Annotation Sensitivity EACL 2024 Rater Cohesion and Quality from a Vicarious Perspective EMNLP 2024 ReproHum #0043-4: Evaluating Summarization Models: investigating the impact of education and language proficiency on reproducibility COLING 2024 Rethinking Human Evaluation Protocol for Text-to-Video Models: Enhancing Reliability, Reproducibility, and Practicality NeurIPS 2024 Sina at FigNews 2024: Multilingual Datasets Annotated with Bias and Propaganda. ACL 2024 Your Model Is Not Predicting Depression Well And That Is Why: A Case Study of PRIMATE Dataset EACL 2024 A Needle in a Haystack: An Analysis of High-Agreement Workers on MTurk for Summarization ACL 2023 Annotating and Training for Population Subjective Views ACL 2023 Comparative evaluation of boundary-relaxed annotation for Entity Linking performance ACL 2023 GATE Teamware 2: An open-source tool for collaborative document classification annotation EACL 2023