Evaluation

Merging Facts, Crafting Fallacies: Evaluating the Contradictory Nature of Aggregated Factual Claims in Long-Form Generations ACL 2024 MetaMetrics-MT: Tuning Meta-Metrics for Machine Translation via Human Preference Calibration EMNLP 2024 MIBench: Evaluating Multimodal Large Language Models over Multiple Images EMNLP 2024 Micro-Bench: A Microscopy Benchmark for Vision-Language Understanding NeurIPS 2024 Mitigating the Impact of Reference Quality on Evaluation of Summarization Systems with Reference-Free Metrics EMNLP 2024 MVP-Bench: Can Large Vision-Language Models Conduct Multi-level Visual Perception Like Humans? EMNLP 2024 “My Answer is C”: First-Token Probabilities Do Not Match Text Answers in Instruction-Tuned Language Models ACL 2024 Navigating the Maze of Explainable AI: A Systematic Approach to Evaluating Methods and Metrics NeurIPS 2024 On Leakage of Code Generation Evaluation Datasets EMNLP 2024 On the Content Bias in Frechet Video Distance CVPR 2024 On the Faithfulness of Vision Transformer Explanations CVPR 2024 On Training Data Influence of GPT Models EMNLP 2024 Pitfalls and Outlooks in Using COMET EMNLP 2024 Please note that I’m just an AI: Analysis of Behavior Patterns of LLMs in (Non-)offensive Speech Identification EMNLP 2024 Plot Twist: Multimodal Models Don’t Comprehend Simple Chart Details EMNLP 2024 POSIX: A Prompt Sensitivity Index For Large Language Models EMNLP 2024 RAGLAB: A Modular and Research-Oriented Unified Framework for Retrieval-Augmented Generation EMNLP 2024 Ranking Large Language Models without Ground Truth ACL 2024 Realistic Evaluation of Toxicity in Large Language Models ACL 2024 Reasoning in Token Economies: Budget-Aware Evaluation of LLM Reasoning Strategies EMNLP 2024 Reasoning or a Semblance of it? A Diagnostic Study of Transitive Reasoning in LLMs EMNLP 2024 Re-Evaluating Evaluation for Multilingual Summarization EMNLP 2024 Rethinking FID: Towards a Better Evaluation Metric for Image Generation CVPR 2024 Revisiting Query Variation Robustness of Transformer Models EMNLP 2024 Safetywashing: Do AI Safety Benchmarks Actually Measure Safety Progress? NeurIPS 2024