benchmark evaluation

SimulBench: Evaluating Language Models with Creative Simulation Tasks NAACL 2025 Sinhala Encoder-only Language Models and Evaluation ACL 2025 SinhalaMMLU: A Comprehensive Benchmark for Evaluating Multitask Language Understanding in Sinhala EMNLP 2025 SITE: towards Spatial Intelligence Thorough Evaluation ICCV 2025 SKA-Bench: A Fine-Grained Benchmark for Evaluating Structured Knowledge Understanding of LLMs EMNLP 2025 SmartBench: Is Your LLM Truly a Good Chinese Smartphone Assistant? EMNLP 2025 Social Bias in Popular Question-Answering Benchmarks IJCNLP 2025 SocialCC: Interactive Evaluation for Cultural Competence in Language Agents ACL 2025 SocialEval: Evaluating Social Intelligence of Large Language Models ACL 2025 SolEval: Benchmarking Large Language Models for Repository-level Solidity Smart Contract Generation EMNLP 2025 Something’s Fishy in the Data Lake: A Critical Re-evaluation of Table Union Search Benchmarks ACL 2025 SOVA-Bench: Benchmarking the Speech Conversation Ability for LLM-based Voice Assistant Interspeech 2025 SPA-BENCH: A COMPREHENSIVE BENCHMARK FOR SMARTPHONE AGENT EVALUATION ICLR 2025 Spatial457: A Diagnostic Benchmark for 6D Spatial Reasoning of Large Mutimodal Models CVPR 2025 SQLSpace: A Representation Space for Text-to-SQL to Discover and Mitigate Robustness Gaps EMNLP 2025 STEER-BENCH: A Benchmark for Evaluating the Steerability of Large Language Models EMNLP 2025 Stress-Testing Multimodal Foundation Models for Crystallographic Reasoning ACL 2025 STRICT: Stress-Test of Rendering Image Containing Text EMNLP 2025 StrucText-Eval: Evaluating Large Language Model’s Reasoning Ability in Structure-Rich Text ACL 2025 StructFact: Reasoning Factual Knowledge from Structured Data with Large Language Models ACL 2025 SubLIME: Subset Selection via Rank Correlation Prediction for Data-Efficient LLM Evaluation ACL 2025 Surge: On the Potential of Large Language Models as General-Purpose Surrogate Code Executors EMNLP 2025 SVLTA: Benchmarking Vision-Language Temporal Alignment via Synthetic Video Situation CVPR 2025 Swan and ArabicMTEB: Dialect-Aware, Arabic-Centric, Cross-Lingual, and Cross-Cultural Embedding Models and Benchmarks NAACL 2025 SWE-MERA: A Dynamic Benchmark for Agenticly Evaluating Large Language Models on Software Engineering Tasks EMNLP 2025