benchmark evaluation

OmniMedVQA: A New Large-Scale Comprehensive Evaluation Benchmark for Medical LVLM CVPR 2024 One Thousand and One Pairs: A “novel” challenge for long-context language models EMNLP 2024 On Leakage of Code Generation Evaluation Datasets EMNLP 2024 On the Evaluation of Speech Foundation Models for Spoken Language Understanding ACL 2024 On the Worst Prompt Performance of Large Language Models NeurIPS 2024 On Train-Test Class Overlap and Detection for Image Retrieval CVPR 2024 OOP: Object-Oriented Programming Evaluation Benchmark for Large Language Models ACL 2024 Open-ended VQA benchmarking of Vision-Language models by exploiting Classification datasets and their semantic hierarchy ICLR 2024 Open Ko-LLM Leaderboard: Evaluating Large Language Models in Korean with Ko-H5 Benchmark ACL 2024 OpenT2T: An Open-Source Toolkit for Table-to-Text Generation EMNLP 2024 OpenToM: A Comprehensive Benchmark for Evaluating Theory-of-Mind Reasoning Capabilities of Large Language Models ACL 2024 OSWorld: Benchmarking Multimodal Agents for Open-Ended Tasks in Real Computer Environments NeurIPS 2024 Overview of the Shared Task on Machine Translation Gender Bias Evaluation with Multilingual Holistic Bias ACL 2024 Parallel Empirical Evaluations: Resilience despite Concurrency AAAI 2024 PARIKSHA: A Large-Scale Investigation of Human-LLM Evaluator Agreement on Multilingual and Multi-Cultural Data EMNLP 2024 Perceptions to Beliefs: Exploring Precursory Inferences for Theory of Mind in Large Language Models EMNLP 2024 PertEval: Unveiling Real Knowledge Capacity of LLMs with Knowledge-Invariant Perturbations NeurIPS 2024 PhonologyBench: Evaluating Phonological Skills of Large Language Models ACL 2024 Planning, Creation, Usage: Benchmarking LLMs for Comprehensive Tool Utilization in Real-World Complex Scenarios ACL 2024 Plausibly Problematic Questions in Multiple-Choice Benchmarks for Commonsense Reasoning EMNLP 2024 Plot Twist: Multimodal Models Don’t Comprehend Simple Chart Details EMNLP 2024 Poor-Supervised Evaluation for SuperLLM via Mutual Consistency ACL 2024 PPTC Benchmark: Evaluating Large Language Models for PowerPoint Task Completion ACL 2024 Privacy Evaluation Benchmarks for NLP Models EMNLP 2024 Probing Language Models for Pre-training Data Detection ACL 2024