benchmark evaluation

1539 papers

Explore in graph

Also known as

MT-BENCH BDC

Co-occurring keywords

large language model (12755) question answering (2904) multimodal learning (4622) language model (4573) multimodal large language model (865) vision-language model (2235) visual question answering (1000) evaluation benchmark (250) multilingual nlp (1423) benchmark dataset (619)

Papers

CDEval: A Benchmark for Measuring the Cultural Dimensions of Large Language Models ACL 2024

Overview of the Shared Task on Machine Translation Gender Bias Evaluation with Multilingual Holistic Bias ACL 2024

Do LLMs Speak Kazakh? A Pilot Evaluation of Seven Models ACL 2024

Schema-Driven Information Extraction from Heterogeneous Tables EMNLP 2024

Factcheck-Bench: Fine-Grained Evaluation Benchmark for Automatic Fact-checkers EMNLP 2024

MVP-Bench: Can Large Vision-Language Models Conduct Multi-level Visual Perception Like Humans? EMNLP 2024

Sonnet or Not, Bot? Poetry Evaluation for Large Models and Datasets EMNLP 2024

MedExQA: Medical Question Answering Benchmark with Multiple Explanations ACL 2024

PhonologyBench: Evaluating Phonological Skills of Large Language Models ACL 2024

MVBench: A Comprehensive Multi-modal Video Understanding Benchmark CVPR 2024

Beyond the Numbers: Transparency in Relation Extraction Benchmark Creation and Leaderboards EMNLP 2024

Unveiling Causal Reasoning in Large Language Models: Reality or Mirage? NIPS 2024

Are LLMs Capable of Data-based Statistical and Causal Reasoning? Benchmarking Advanced Quantitative Reasoning with Data ACL 2024

GlitchBench: Can Large Multimodal Models Detect Video Game Glitches? CVPR 2024

Advancement in Graph Understanding: A Multimodal Benchmark and Fine-Tuning of Vision-Language Models ACL 2024

Plausibly Problematic Questions in Multiple-Choice Benchmarks for Commonsense Reasoning EMNLP 2024

When Benchmarks are Targets: Revealing the Sensitivity of Large Language Model Leaderboards ACL 2024

VHELM: A Holistic Evaluation of Vision Language Models NIPS 2024

MMLU-Pro: A More Robust and Challenging Multi-Task Language Understanding Benchmark NIPS 2024

Towards a Complete Benchmark on Video Moment Localization AISTATS 2024

EU DisinfoTest: a Benchmark for Evaluating Language Models’ Ability to Detect Disinformation Narratives EMNLP 2024

FLORES+ Translation and Machine Translation Evaluation for the Erzya Language EMNLP 2024

Large Language Models Are Not Strong Abstract Reasoners IJCAI 2024

GRASP: A Novel Benchmark for Evaluating Language GRounding and Situated Physics Understanding in Multimodal Language Models IJCAI 2024

Reframing Spatial Reasoning Evaluation in Language Models: A Real-World Simulation Benchmark for Qualitative Reasoning IJCAI 2024