conftrace_

annotation quality

109 papers

Explore in graph

Co-occurring keywords

inter-annotator agreement (134) human evaluation (379) large language model (13587) named entity recognition (2813) text classification (6793) data quality (130) data annotation (101) text annotation (151) natural language processing (2034) active learning (1301)

Papers

A Scalable Framework for Automated NER Annotation Correction in Low-Resource Languages EACL 2026
LLMs as Span Annotators: A Comparative Study of LLMs and Humans EACL 2026
MQM Re-Annotation: A Technique for Collaborative Evaluation of Machine Translation ACL 2026
IDEAlign: Comparing Ideas of Large Language Models to Domain Experts EACL 2026
Next Generation Active Learning: Mixture of LLMs in the Loop AAAI 2026
Do Large Language Models Reflect Demographic Pluralism in Safety? EACL 2026
Annotating the Annotators: Analysis, Insights and Modelling from an Annotation Campaign on Persuasion Techniques Detection ACL 2025
Can Third Parties Read Our Emotions? ACL 2025
On the Role of Speech Data in Reducing Toxicity Detection Bias NAACL 2025
Consistency is Key: Disentangling Label Variation in Natural Language Processing with Intra-Annotator Agreement EMNLP 2025
Humans Hallucinate Too: Language Models Identify and Correct Subjective Annotation Errors With Label-in-a-Haystack Prompts EMNLP 2025
Challenges in Trustworthy Human Evaluation of Chatbots NAACL 2025
AutoComPose: Automatic Generation of Pose Transition Descriptions for Composed Pose Retrieval Using Multimodal LLMs ICCV 2025
Evaluating the Quality of Benchmark Datasets for Low-Resource Languages: A Case Study on Turkish ACL 2025
The Alternative Annotator Test for LLM-as-a-Judge: How to Statistically Justify Replacing Human Annotators with LLMs ACL 2025
LLMs as annotators of argumentation EMNLP 2025
Task Configuration Impacts Annotation Quality and Model Training Performance in Crowdsourced Image Segmentation WACV 2025
Balancing Quality and Variation: Spam Filtering Distorts Data Label Distributions EMNLP 2025
Can External Validation Tools Improve Annotation Quality for LLM-as-a-Judge? ACL 2025
Beyond Benchmarks: Building a Richer Cross-Document Event Coreference Dataset with Decontextualization NAACL 2025
Strong Empowered and Aligned Weak Mastered Annotation for Weak-to-Strong Generalization AAAI 2025
ARTICLE: Annotator Reliability Through In-Context Learning (Student Abstract) AAAI 2025
CrowdAgent: Multi-Agent Managed Multi-Source Annotation System EMNLP 2025
Are LLMs Better than Reported? Detecting Label Errors and Mitigating Their Effect on Model Performance EMNLP 2025
Where it’s at: Annotating Verb Placement Types in Learner Language ACL 2025