conftrace_

inter-rater reliability

13 papers

Explore in graph

Also known as

IRR

Co-occurring keywords · papers mentioning both

annotation quality (4) large language model (3) text classification (3) statistical analysis (3) machine translation (2) qualitative analysis (1) automated annotation (1) mental health (1) argument annotation (1) semi-automatic annotation (1)

Papers

Development and Benchmarking of a Blended Human-AI Qualitative Research Assistant ACL 2026 Forest vs Tree: The (N, K) Trade-off in Reproducible ML Evaluation AAAI 2026 When Can We Trust LLMs in Mental Health? Large-Scale Benchmarks for Reliable LLM Evaluation EACL 2026 Can Large Language Models Outperform Non-Experts in Poetry Evaluation? A Comparative Study Using the Consensual Assessment Technique EMNLP 2025 Efficient Annotator Reliability Assessment with EffiARA ACL 2025 Finding Replicable Human Evaluations via Stable Ranking Probability NAACL 2024 SIGHT: A Large Annotated Dataset on Student Insights Gathered from Higher Education Transcripts ACL 2023 k-Rater Reliability: The Correct Unit of Reliability for Aggregated Human Annotations ACL 2022 Cross-replication Reliability - An Empirical Approach to Interpreting Inter-rater Reliability ACL 2021 An Evaluation of Manual and Semi-Automatic Laughter Annotation Interspeech 2020 Rubric Reliability and Annotation of Content and Argument in Source-Based Argument Essays ACL 2019 SMART: An Open Source Data Labeling Platform for Supervised Learning JMLR 2019 Improving Evaluation of Document-level Machine Translation Quality Estimation EACL 2017