annotation quality

A Scalable Framework for Automated NER Annotation Correction in Low-Resource Languages EACL 2026 Do Large Language Models Reflect Demographic Pluralism in Safety? EACL 2026 IDEAlign: Comparing Ideas of Large Language Models to Domain Experts EACL 2026 LLMs as Span Annotators: A Comparative Study of LLMs and Humans EACL 2026 MQM Re-Annotation: A Technique for Collaborative Evaluation of Machine Translation ACL 2026 Next Generation Active Learning: Mixture of LLMs in the Loop AAAI 2026 Annotating the Annotators: Analysis, Insights and Modelling from an Annotation Campaign on Persuasion Techniques Detection ACL 2025 Are LLMs Better than Reported? Detecting Label Errors and Mitigating Their Effect on Model Performance EMNLP 2025 ARTICLE: Annotator Reliability Through In-Context Learning (Student Abstract) AAAI 2025 A Survey on Automated Distractor Evaluation in Multiple-Choice Tasks ACL 2025 AutoComPose: Automatic Generation of Pose Transition Descriptions for Composed Pose Retrieval Using Multimodal LLMs ICCV 2025 Balancing Quality and Variation: Spam Filtering Distorts Data Label Distributions EMNLP 2025 Beyond Benchmarks: Building a Richer Cross-Document Event Coreference Dataset with Decontextualization NAACL 2025 Can External Validation Tools Improve Annotation Quality for LLM-as-a-Judge? ACL 2025 Can Third Parties Read Our Emotions? ACL 2025 Challenges in Trustworthy Human Evaluation of Chatbots NAACL 2025 Cheap Annotation of Complex Information: A Study on the Annotation of Information Status in German TEDx Talks ACL 2025 Consistency is Key: Disentangling Label Variation in Natural Language Processing with Intra-Annotator Agreement EMNLP 2025 CrowdAgent: Multi-Agent Managed Multi-Source Annotation System EMNLP 2025 Efficient Annotator Reliability Assessment with EffiARA ACL 2025 Evaluating the Quality of Benchmark Datasets for Low-Resource Languages: A Case Study on Turkish ACL 2025 Humans Hallucinate Too: Language Models Identify and Correct Subjective Annotation Errors With Label-in-a-Haystack Prompts EMNLP 2025 LLMs as annotators of argumentation EMNLP 2025 On the Role of Speech Data in Reducing Toxicity Detection Bias NAACL 2025 Refined Assessment for Translation Evaluation: Rethinking Machine Translation Evaluation in the Era of Human-Level Systems EMNLP 2025