Evaluation

SimBA: Simplifying Benchmark Analysis Using Performance Matrices Alone EMNLP 2025 SimulatorArena: Are User Simulators Reliable Proxies for Multi-Turn Evaluation of AI Assistants? EMNLP 2025 Single Ground Truth Is Not Enough: Adding Flexibility to Aspect-Based Sentiment Analysis Evaluation NAACL 2025 SinhalaMMLU: A Comprehensive Benchmark for Evaluating Multitask Language Understanding in Sinhala EMNLP 2025 SITE: towards Spatial Intelligence Thorough Evaluation ICCV 2025 skLEP: A Slovak General Language Understanding Benchmark ACL 2025 SocialEval: Evaluating Social Intelligence of Large Language Models ACL 2025 SocioBench: Modeling Human Behavior in Sociological Surveys with Large Language Models EMNLP 2025 Something’s Fishy in the Data Lake: A Critical Re-evaluation of Table Union Search Benchmarks ACL 2025 Sources of Disagreement in Data for LLM Instruction Tuning COLING 2025 SPaRC: A Spatial Pathfinding Reasoning Challenge EMNLP 2025 Specification-Aware Machine Translation and Evaluation for Purpose Alignment EMNLP 2025 SPECS: Specificity-Enhanced CLIP-Score for Long Image Caption Evaluation EMNLP 2025 SpeechIQ: Speech-Agentic Intelligence Quotient Across Cognitive Levels in Voice Understanding by Large Language Models ACL 2025 SQLWOZ: A Realistic Task-Oriented Dialogue Dataset with SQL-Based Dialogue State Representation for Complex User Requirements EMNLP 2025 SSA-COMET: Do LLMs Outperform Learned Metrics in Evaluating MT for Under-Resourced African Languages? EMNLP 2025 SSA: Semantic Contamination of LLM-Driven Fake News Detection EMNLP 2025 Standard Quality Criteria Derived from Current NLP Evaluations for Guiding Evaluation Design and Grounding Comparability and AI Compliance Assessments ACL 2025 STEER-BENCH: A Benchmark for Evaluating the Steerability of Large Language Models EMNLP 2025 Stop Playing the Guessing Game! Evaluating Conversational Recommender Systems via Target-free User Simulation EMNLP 2025 Stress-Testing Multimodal Foundation Models for Crystallographic Reasoning ACL 2025 StrucText-Eval: Evaluating Large Language Model’s Reasoning Ability in Structure-Rich Text ACL 2025 “Stupid robot, I want to speak to a human!” User Frustration Detection in Task-Oriented Dialog Systems COLING 2025 SubLIME: Subset Selection via Rank Correlation Prediction for Data-Efficient LLM Evaluation ACL 2025 SVLTA: Benchmarking Vision-Language Temporal Alignment via Synthetic Video Situation CVPR 2025