conftrace_

annotation quality

109 papers

Explore in graph

Co-occurring keywords

inter-annotator agreement (134) human evaluation (379) large language model (13587) named entity recognition (2813) text classification (6793) data quality (130) data annotation (101) text annotation (151) natural language processing (2034) active learning (1301)

Papers

ARTICLE: Annotator Reliability Through In-Context Learning (Student Abstract) AAAI 2025
Are LLMs Better than Reported? Detecting Label Errors and Mitigating Their Effect on Model Performance EMNLP 2025
AutoComPose: Automatic Generation of Pose Transition Descriptions for Composed Pose Retrieval Using Multimodal LLMs ICCV 2025
Evaluating the Quality of Benchmark Datasets for Low-Resource Languages: A Case Study on Turkish ACL 2025
Efficient Online Crowdsourcing with Complex Annotations AAAI 2024
Geo-Cultural Representation and Inclusion in Language Technologies COLING 2024
ReproHum #0043-4: Evaluating Summarization Models: investigating the impact of education and language proficiency on reproducibility COLING 2024
“Get Their Hands Dirty, Not Mine”: On Researcher-Annotator Collaboration and the Agency of Annotators ACL 2024
Automatic Detection and Labelling of Personal Data in Case Reports from the ECHR in Spanish: Evaluation of Two Different Annotation Approaches EACL 2024
A Gold Standard with Silver Linings: Scaling Up Annotation for Distinguishing Bosnian, Croatian, Montenegrin and Serbian COLING 2024
Context Does Matter: Implications for Crowdsourced Evaluation Labels in Task-Oriented Dialogue Systems NAACL 2024
On Efficient and Statistical Quality Estimation for Data Annotation ACL 2024
Estimating the Level of Dialectness Predicts Inter-annotator Agreement in Multi-dialect Arabic Datasets ACL 2024
Narrative Navigators at FIGNEWS 2024 Shared Task: New Frontiers in Bias and Propaganda Annotation Techniques ACL 2024
Order Effects in Annotation Tasks: Further Evidence of Annotation Sensitivity EACL 2024
Sina at FigNews 2024: Multilingual Datasets Annotated with Bias and Propaganda. ACL 2024
Your Model Is Not Predicting Depression Well And That Is Why: A Case Study of PRIMATE Dataset EACL 2024
Beyond Human-Only: Evaluating Human-Machine Collaboration for Collecting High-Quality Translation Data EMNLP 2024
Rethinking Human Evaluation Protocol for Text-to-Video Models: Enhancing Reliability, Reproducibility, and Practicality NIPS 2024
Rater Cohesion and Quality from a Vicarious Perspective EMNLP 2024
Finding Replicable Human Evaluations via Stable Ranking Probability NAACL 2024
Multi-label and Multi-target Sampling of Machine Annotation for Computational Stance Detection EMNLP 2023
How Good Is the Model in Model-in-the-loop Event Coreference Resolution Annotation? ACL 2023
Incorporating Worker Perspectives into MTurk Annotation Practices for NLP EMNLP 2023
K-HATERS: A Hate Speech Detection Corpus in Korean with Target-Specific Ratings EMNLP 2023