human annotation

Can Reasoning Help Large Language Models Capture Human Annotator Disagreement? EACL 2026 CommonLID: Re-evaluating State-of-the-Art Language Identification Performance on Web Data ACL 2026 Forest vs Tree: The (N, K) Trade-off in Reproducible ML Evaluation AAAI 2026 Accelerating Unbiased LLM Evaluation via Synthetic Feedback ICML 2025 A Little Human Data Goes A Long Way ACL 2025 “All that Glitters”: Techniques for Evaluations with Unreliable Model and Human Annotations NAACL 2025 AmbiStory: A Challenging Dataset of Lexically Ambiguous Short Stories EMNLP 2025 Are Humans as Brittle as Large Language Models? IJCNLP 2025 Bridging the Socioeconomic Gap in Education: A Hybrid AI and Human Annotation Approach CoNLL 2025 Can LLMs Rank the Harmfulness of Smaller LLMs? We are Not There Yet ACL 2025 Can Unconfident LLM Annotations Be Used for Confident Conclusions? NAACL 2025 Chain-of-Talkers (CoTalk): Fast Human Annotation of Dense Image Captions EMNLP 2025 Cheems: A Practical Guidance for Building and Evaluating Chinese Reward Models from Scratch ACL 2025 CiteEval: Principle-Driven Citation Evaluation for Source Attribution ACL 2025 CNNSum: Exploring Long-Context Summarization with Large Language Models in Chinese Novels ACL 2025 Did the Writer Actually Visit the Location? Analysis of Location Reviews from Visit Experience IJCNLP 2025 Evaluating WMT 2025 Metrics Shared Task Submissions on the SSA-MTE African Challenge Set EMNLP 2025 “Feels Feminine to Me”: Understanding Perceived Gendered Style through Human Annotations EMNLP 2025 Findings of the First Patent Claims Translation Task at WAT2025 AACL 2025 IndoPref: A Multi-Domain Pairwise Preference Dataset for Indonesian IJCNLP 2025 Just Put a Human in the Loop? Investigating LLM-Assisted Annotation for Subjective Tasks ACL 2025 LLMs as Medical Safety Judges: Evaluating Alignment with Human Annotation in Patient-Facing QA ACL 2025 Measuring Label Ambiguity in Subjective Tasks using Predictive Uncertainty Estimation ACL 2025 OpenING: A Comprehensive Benchmark for Judging Open-ended Interleaved Image-Text Generation CVPR 2025 PLLuM-Align: Polish Preference Dataset for Large Language Model Alignment EMNLP 2025