Evaluation

SYNC: A Synthetic Long-Context Understanding Benchmark for Controlled Comparisons of Model Capabilities EMNLP 2025 Synthetic Data for Evaluation: Supporting LLM-as-a-Judge Workflows with EvalAssist EMNLP 2025 SynthTextEval: Synthetic Text Data Generation and Evaluation for High-Stakes Domains EMNLP 2025 T2V-CompBench: A Comprehensive Benchmark for Compositional Text-to-video Generation CVPR 2025 TableBench: A Comprehensive and Complex Benchmark for Table Question Answering AAAI 2025 Table Understanding and (Multimodal) LLMs: A Cross-Domain Case Study on Scientific vs. Non-Scientific Data ACL 2025 TASER: Translation Assessment via Systematic Evaluation and Reasoning EMNLP 2025 Temporal-Aware Evaluation and Learning for Temporal Graph Neural Networks AAAI 2025 Testing the Boundaries of LLMs: Dialectal and Language-Variety Tasks COLING 2025 TFDP: Token-Efficient Disparity Audits for Autoregressive LLMs via Single-Token Masked Evaluation EMNLP 2025 Thai Winograd Schemas: A Benchmark for Thai Commonsense Reasoning COLING 2025 That doesn’t sound right: Evaluating speech transcription quality in field linguistics corpora ACL 2025 The 2025 ReproNLP Shared Task on Reproducibility of Evaluations in NLP: Overview and Results ACL 2025 The Alternative Annotator Test for LLM-as-a-Judge: How to Statistically Justify Replacing Human Annotators with LLMs ACL 2025 The Effectiveness of Curvature-Based Rewiring and the Role of Hyperparameters in GNNs Revisited ICLR 2025 The Emperor’s New Reasoning: Format Imitation Overshadows Genuine Mathematical Understanding in SFT EMNLP 2025 The Fellowship of the LLMs: Multi-Model Workflows for Synthetic Preference Optimization Dataset Generation ACL 2025 The First Workshop on Multilingual Counterspeech Generation at COLING 2025: Overview of the Shared Task COLING 2025 The Illusion of Progress: Re-evaluating Hallucination Detection in LLMs EMNLP 2025 The Million Authors Corpus: A Cross-Lingual and Cross-Domain Wikipedia Dataset for Authorship Verification ACL 2025 The Mirage of Model Editing: Revisiting Evaluation in the Wild ACL 2025 Theory of Mind in Large Language Models: Assessment and Enhancement ACL 2025 The UNLP 2025 Shared Task on Detecting Social Media Manipulation ACL 2025 The Unreasonable Effectiveness of Open Science in AI: A Replication Study AAAI 2025 Thinking Racial Bias in Fair Forgery Detection: Models, Datasets and Evaluations AAAI 2025