benchmark evaluation

1539 papers

Explore in graph

Also known as

MT-BENCH BDC

Co-occurring keywords

large language model (12755) question answering (2904) multimodal learning (4622) language model (4573) multimodal large language model (865) vision-language model (2235) visual question answering (1000) evaluation benchmark (250) multilingual nlp (1423) benchmark dataset (619)

Papers

MCP-AgentBench: Evaluating Real-World Language Agent Performance with MCP-Mediated Tools AAAI 2026

ConInstruct: Evaluating Large Language Models on Conflict Detection and Resolution in Instructions AAAI 2026

HPSU: A Benchmark for Human-Level Perception in Real-World Spoken Speech Understanding AAAI 2026

Res-Bench: Benchmarking the Robustness of Multimodal Large Language Models to Dynamic Resolution Input AAAI 2026

Small Models Exhibit Limited Answer Consistency in Repetition Trials of the Multiple-Choice MMLU-Redux and MedQA Benchmarks AAAI 2026

Are We on the Right Way to Assess Document Retrieval-Augmented Generation? AAAI 2026

OmniBench: A Comprehensive Benchmark Integrating Real-World, Time-sensitive, and Multi-Hop Questions with a Multi-Dimensional Hybrid Evaluation Framework AAAI 2026

MetaEval: Measuring the Discrimination of Benchmarks for Efficient LLM Evaluation AAAI 2026

Reasoning or Memorization? Unreliable Results of Reinforcement Learning Due to Data Contamination AAAI 2026

DeepPhy: Benchmarking Agentic VLMs on Physical Reasoning AAAI 2026

Mind the Third Eye! Benchmarking Privacy Awareness in MLLM-powered Smartphone Agents AAAI 2026

Bridging the Copyright Gap: Do Large Vision-Language Models Recognize and Respect Copyrighted Content? AAAI 2026

DarkBench+: An Extended Benchmark for Evaluating Dark Patterns in Large Language Models AAAI 2026

Benchmarking Trustworthiness in Multimodal LLMs for Video Understanding AAAI 2026

CARE-Bench: A Benchmark of Diverse Client Simulations Guided by Expert Principles for Evaluating LLMs in Psychological Counseling AAAI 2026

QualBench: Benchmarking Chinese LLMs with Localized Professional Qualifications for Vertical Domain Evaluation EMNLP 2025

OmniEval: An Omnidirectional and Automatic RAG Evaluation Benchmark in Financial Domain EMNLP 2025

LMR-BENCH: Evaluating LLM Agent’s Ability on Reproducing Language Modeling Research EMNLP 2025

LLMs instead of Human Judges? A Large Scale Empirical Study across 20 NLP Evaluation Tasks ACL 2025

Diffusion Models Through a Global Lens: Are They Culturally Inclusive? ACL 2025

T2V-CompBench: A Comprehensive Benchmark for Compositional Text-to-video Generation CVPR 2025

Identifying Open Challenges in Language Identification ACL 2025

FinNLP-FNP-LLMFinLegal-2025 Shared Task: Financial Misinformation Detection Challenge Task COLING 2025

WinoWhat: A Parallel Corpus of Paraphrased WinoGrande Sentences with Common Sense Categorization CONLL 2025

ExpliCa: Evaluating Explicit Causal Reasoning in Large Language Models ACL 2025