conftrace_

annotation quality

109 papers

Explore in graph

Co-occurring keywords

inter-annotator agreement (134) human evaluation (379) large language model (13587) named entity recognition (2813) text classification (6793) data quality (130) data annotation (101) text annotation (151) natural language processing (2034) active learning (1301)

Papers

Unveiling the Multi-Annotation Process: Examining the Influence of Annotation Quantity and Instance Difficulty on Model Performance EMNLP 2023
eevvgg at SemEval-2023 Task 11: Offensive Language Classification with Rater-based Information ACL 2023
Incorporating Worker Perspectives into MTurk Annotation Practices for NLP EMNLP 2023
Comparative evaluation of boundary-relaxed annotation for Entity Linking performance ACL 2023
How Good Is the Model in Model-in-the-loop Event Coreference Resolution Annotation? ACL 2023
Verifying Annotation Agreement without Multiple Experts: A Case Study with Gujarati SNACS ACL 2023
Rethinking Annotation: Can Language Learners Contribute? ACL 2023
A Needle in a Haystack: An Analysis of High-Agreement Workers on MTurk for Summarization ACL 2023
Annotating and Training for Population Subjective Views ACL 2023
K-HATERS: A Hate Speech Detection Corpus in Korean with Target-Specific Ratings EMNLP 2023
DiaBiz.Kom - towards a Polish Dialogue Act Corpus Based on ISO 24617-2 Standard COLING 2022
Establishing Annotation Quality in Multi-label Annotations COLING 2022
The “Problem” of Human Label Variation: On Ground Truth in Data, Modeling and Evaluation EMNLP 2022
Inter-annotator agreement is not the ceiling of machine learning performance: Evidence from a comprehensive set of simulations ACL 2022
Toward More Meaningful Resources for Lower-resourced Languages ACL 2022
Does Recommend-Revise Produce Reliable Annotations? An Analysis on Missing Instances in DocRED ACL 2022
On the Diversity and Limits of Human Explanations NAACL 2022
Cascading Biases: Investigating the Effect of Heuristic Annotation Strategies on Data and Models EMNLP 2022
GENIE: Toward Reproducible and Standardized Human Evaluation for Text Generation EMNLP 2022
On reporting scores and agreement for error annotation tasks EMNLP 2022
Evaluating Diversity of Multiword Expressions in Annotated Text COLING 2022
k-Rater Reliability: The Correct Unit of Reliability for Aggregated Human Annotations ACL 2022
Tools Impact on the Quality of Annotations for Chat Untangling IJCNLP 2021
Adaptiope: A Modern Benchmark for Unsupervised Domain Adaptation WACV 2021
Cross-replication Reliability - An Empirical Approach to Interpreting Inter-rater Reliability IJCNLP 2021