conftrace_

human annotation

115 papers

Explore in graph

Co-occurring keywords

large language model (13587) active learning (1301) text classification (6793) natural language processing (2034) human evaluation (379) language model (4599) model evaluation (445) multilingual nlp (1426) benchmark dataset (635) natural language inference (1287)

Papers

IndoPref: A Multi-Domain Pairwise Preference Dataset for Indonesian AACL 2025
Measuring Label Ambiguity in Subjective Tasks using Predictive Uncertainty Estimation ACL 2025
Bridging the Socioeconomic Gap in Education: A Hybrid AI and Human Annotation Approach ACL 2025
Can LLMs Rank the Harmfulness of Smaller LLMs? We are Not There Yet ACL 2025
LLMs as Medical Safety Judges: Evaluating Alignment with Human Annotation in Patient-Facing QA ACL 2025
Benchmarking Hindi LLMs: A New Suite of Datasets and a Comparative Analysis IJCNLP 2025
“All that Glitters”: Techniques for Evaluations with Unreliable Model and Human Annotations NAACL 2025
SeeGULL Multilingual: a Dataset of Geo-Culturally Situated Stereotypes ACL 2024
GPT is Not an Annotator: The Necessity of Human Annotation in Fairness Benchmark Construction ACL 2024
HQP: A Human-Annotated Dataset for Detecting Online Propaganda ACL 2024
Speech vs. Transcript: Does It Matter for Human Annotators in Speech Summarization? ACL 2024
STAR: SocioTechnical Approach to Red Teaming Language Models EMNLP 2024
Meta-Evaluation of Sentence Simplification Metrics COLING 2024
Diversity-Aware Annotation for Conversational AI Safety COLING 2024
The Effectiveness of LLMs as Annotators: A Comparative Overview and Empirical Analysis of Direct Representation COLING 2024
A Survey on Modelling Morality for Text Analysis ACL 2024
Characterizing Human and Zero-Shot GPT-3.5 Object-Similarity Judgments NAACL 2024
An Interactive Human-Machine Learning Interface for Collecting and Learning from Complex Annotations IJCAI 2024
Order Effects in Annotation Tasks: Further Evidence of Annotation Sensitivity EACL 2024
Active preference learning for ordering items in- and out-of-sample NIPS 2024
How Much Annotation is Needed to Compare Summarization Models? NAACL 2024
Enhancing Text Classification through LLM-Driven Active Learning and Human Annotation EACL 2024
AHA: Human-Assisted Out-of-Distribution Generalization and Detection NIPS 2024
LLM-Rubric: A Multidimensional, Calibrated Approach to Automated Evaluation of Natural Language Texts ACL 2024
Annotator-Centric Active Learning for Subjective NLP Tasks EMNLP 2024