multimodal evaluation

GAICo: Demonstrating a Unified Framework for Multi-Modal GenAI Evaluation AAAI 2026 Can’t See the Forest for the Trees: Benchmarking Multimodal Safety Awareness for Multimodal LLMs ACL 2025 Can Vision Language Models Understand Mimed Actions? ACL 2025 CliME: Evaluating Multimodal Climate Discourse on Social Media and the Climate Alignment Quotient (CAQ) ACL 2025 FinMME: Benchmark Dataset for Financial Multi-Modal Reasoning Evaluation ACL 2025 FlagEvalMM: A Flexible Framework for Comprehensive Multimodal Model Evaluation ACL 2025 Guiding Vision-Language Model Selection for Visual Question-Answering Across Tasks, Domains, and Knowledge Types COLING 2025 IntelliCockpitBench: A Comprehensive Benchmark to Evaluate VLMs for Intelligent Cockpit ACL 2025 IRR: Image Review Ranking Framework for Evaluating Vision-Language Models COLING 2025 MedXpertQA: Benchmarking Expert-Level Medical Reasoning and Understanding ICML 2025 MEGA-Bench: Scaling Multimodal Evaluation to over 500 Real-World Tasks ICLR 2025 MotionBench: Benchmarking and Improving Fine-grained Video Motion Understanding for Vision Language Models CVPR 2025 OmnixR: Evaluating Omni-modality Language Models on Reasoning across Modalities ICLR 2025 Pay More Attention to Images: Numerous Images-Oriented Multimodal Summarization NAACL 2025 Q-Eval-100K: Evaluating Visual Quality and Alignment Level for Text-to-Vision Content CVPR 2025 Retention Score: Quantifying Jailbreak Risks for Vision Language Models AAAI 2025 SVLTA: Benchmarking Vision-Language Temporal Alignment via Synthetic Video Situation CVPR 2025 Video-MME: The First-Ever Comprehensive Evaluation Benchmark of Multi-modal LLMs in Video Analysis CVPR 2025 Are We on the Right Way for Evaluating Large Vision-Language Models? NeurIPS 2024 ColorSwap: A Color and Word Order Dataset for Multimodal Evaluation ACL 2024 GMAI-MMBench: A Comprehensive Multimodal Evaluation Benchmark Towards General Medical AI NeurIPS 2024 Evaluating Object Hallucination in Large Vision-Language Models EMNLP 2023 What You See is What You Read? Improving Text-Image Alignment Evaluation NeurIPS 2023 Mutual Information Divergence: A Unified Metric for Multimodal Generative Models NeurIPS 2022 Understanding ME? Multimodal Evaluation for Fine-grained Visual Commonsense EMNLP 2022