benchmark evaluation

1539 papers

Explore in graph

Also known as

MT-BENCH BDC

Co-occurring keywords

large language model (12755) question answering (2904) multimodal learning (4622) language model (4573) multimodal large language model (865) vision-language model (2235) visual question answering (1000) evaluation benchmark (250) multilingual nlp (1423) benchmark dataset (619)

Papers

Seemingly Plausible Distractors in Multi-Hop Reasoning: Are Large Language Models Attentive Readers? EMNLP 2024

I Could’ve Asked That: Reformulating Unanswerable Questions EMNLP 2024

STOP! Benchmarking Large Language Models with Sensitivity Testing on Offensive Progressions EMNLP 2024

When Reasoning Meets Information Aggregation: A Case Study with Sports Narratives EMNLP 2024

Do Large Language Models Know How Much They Know? EMNLP 2024

FRoG: Evaluating Fuzzy Reasoning of Generalized Quantifiers in LLMs EMNLP 2024

MAgIC: Investigation of Large Language Model Powered Multi-Agent in Cognition, Adaptability, Rationality and Collaboration EMNLP 2024

PARIKSHA: A Large-Scale Investigation of Human-LLM Evaluator Agreement on Multilingual and Multi-Cultural Data EMNLP 2024

LawBench: Benchmarking Legal Knowledge of Large Language Models EMNLP 2024

AssistantBench: Can Web Agents Solve Realistic and Time-Consuming Tasks? EMNLP 2024

DataTales: A Benchmark for Real-World Intelligent Data Narration EMNLP 2024

Repairs in a Block World: A New Benchmark for Handling User Corrections with Multi-Modal Language Models EMNLP 2024

A Neuro-Symbolic Benchmark Suite for Concept Quality and Reasoning Shortcuts NIPS 2024

Deciphering the Impact of Pretraining Data on Large Language Models through Machine Unlearning ACL 2024

Wikimedia data for AI: a review of Wikimedia datasets for NLP tasks and AI-assisted editing EMNLP 2024

Uncovering What Why and How: A Comprehensive Benchmark for Causation Understanding of Video Anomaly CVPR 2024

Evaluating Numerical Reasoning in Text-to-Image Models NIPS 2024

Categorical Syllogisms Revisited: A Review of the Logical Reasoning Abilities of LLMs for Analyzing Categorical Syllogisms EMNLP 2024

SWE-agent: Agent-Computer Interfaces Enable Automated Software Engineering NIPS 2024

MERA: A Comprehensive LLM Evaluation in Russian ACL 2024

Can Large Multimodal Models Uncover Deep Semantics Behind Images? ACL 2024

WildVision: Evaluating Vision-Language Models in the Wild with Human Preferences NIPS 2024

Evaluating Multilingual Long-Context Models for Retrieval and Reasoning EMNLP 2024

Poor-Supervised Evaluation for SuperLLM via Mutual Consistency ACL 2024

StableToolBench: Towards Stable Large-Scale Benchmarking on Tool Learning of Large Language Models ACL 2024