benchmark evaluation

1539 papers

Explore in graph

Also known as

MT-BENCH BDC

Co-occurring keywords

large language model (12755) question answering (2904) multimodal learning (4622) language model (4573) multimodal large language model (865) vision-language model (2235) visual question answering (1000) evaluation benchmark (250) multilingual nlp (1423) benchmark dataset (619)

Papers

Mind the Gap: Static and Interactive Evaluations of Large Audio Models ACL 2025

CodeRAG-Bench: Can Retrieval Augment Code Generation? NAACL 2025

MILU: A Multi-task Indic Language Understanding Benchmark NAACL 2025

Q-Bench-Video: Benchmark the Video Quality Understanding of LMMs CVPR 2025

Do Large Language Models Know Folktales? A Case Study of Yokai in Japanese Folktales ACL 2025

NTSEBENCH: Cognitive Reasoning Benchmark for Vision Language Models NAACL 2025

BenNumEval: A Benchmark to Assess LLMs’ Numerical Reasoning Capabilities in Bengali ACL 2025

OpenMIBOOD: Open Medical Imaging Benchmarks for Out-Of-Distribution Detection CVPR 2025

VLind-Bench: Measuring Language Priors in Large Vision-Language Models NAACL 2025

OVO-Bench: How Far is Your Video-LLMs from Real-World Online Video Understanding? CVPR 2025

MADial-Bench: Towards Real-world Evaluation of Memory-Augmented Dialogue Generation NAACL 2025

CheckManual: A New Challenge and Benchmark for Manual-based Appliance Manipulation CVPR 2025

Swan and ArabicMTEB: Dialect-Aware, Arabic-Centric, Cross-Lingual, and Cross-Cultural Embedding Models and Benchmarks NAACL 2025

All Languages Matter: Evaluating LMMs on Culturally Diverse 100 Languages CVPR 2025

MLLM-Bench: Evaluating Multimodal LLMs with Per-sample Criteria NAACL 2025

SVLTA: Benchmarking Vision-Language Temporal Alignment via Synthetic Video Situation CVPR 2025

MMAU: A Holistic Benchmark of Agent Capabilities Across Diverse Domains NAACL 2025

Recitation over Reasoning: How Cutting-Edge Language Models Can Fail on Elementary School-Level Reasoning Problems? IJCNLP 2025

ThaiOCRBench: A Task-Diverse Benchmark for Vision-Language Understanding in Thai IJCNLP 2025

Doppelganger-JC: Benchmarking the LLMs’ Understanding of Cross-Lingual Homographs between Japanese and Chinese IJCNLP 2025

What Would You Ask When You First Saw a2+b2=c2? Evaluating LLM on Curiosity-Driven Question Generation IJCNLP 2025

ReasoningWeekly: A General Knowledge and Verbal Reasoning Challenge for Large Language Models IJCNLP 2025

VariantBench: A Framework for Evaluating LLMs on Justifications for Genetic Variant Interpretation IJCNLP 2025

AnaToM: A Dataset Generation Framework for Evaluating Theory of Mind Reasoning Toward the Anatomy of Difficulty through Structurally Controlled Story Generation IJCNLP 2025

LIME: Less Is More for MLLM Evaluation ACL 2025