Evaluation

UOUO: Uncontextualized Uncommon Objects for Measuring Knowledge Horizons of Vision Language Models EMNLP 2024 ValueBench: Towards Comprehensively Evaluating Value Orientations and Understanding of Large Language Models ACL 2024 VarBench: Robust Language Model Benchmarking Through Dynamic Variable Perturbation EMNLP 2024 Variable Importance in High-Dimensional Settings Requires Grouping AAAI 2024 VBench: Comprehensive Benchmark Suite for Video Generative Models CVPR 2024 VeriScore: Evaluating the factuality of verifiable claims in long-form text generation EMNLP 2024 Vertical Validation: Evaluating Implicit Generative Models for Graphs on Thin Support Regions UAI 2024 WalledEval: A Comprehensive Safety Evaluation Toolkit for Large Language Models EMNLP 2024 WaterBench: Towards Holistic Evaluation of Watermarks for Large Language Models ACL 2024 Weak-eval-Strong: Evaluating and Eliciting Lateral Thinking of LLMs with Situation Puzzles NeurIPS 2024 WelQrate: Defining the Gold Standard in Small Molecule Drug Discovery Benchmarking NeurIPS 2024 WenMind: A Comprehensive Benchmark for Evaluating Large Language Models in Chinese Classical Literature and Language Arts NeurIPS 2024 What Are the Odds? Language Models Are Capable of Probabilistic Reasoning EMNLP 2024 What is lost in Normalization? Exploring Pitfalls in Multilingual ASR Model Evaluations EMNLP 2024 What Kind of Sourcery is This? Evaluating GPT-4’s Performance on Linking Scientific Fact to Citations EMNLP 2024 What's Outside the Intersection? Fine-Grained Error Analysis for Semantic Segmentation Beyond IoU WACV 2024 When ”A Helpful Assistant” Is Not Really Helpful: Personas in System Prompts Do Not Improve Performances of Large Language Models EMNLP 2024 When Are Two Lists Better than One?: Benefits and Harms in Joint Decision-Making AAAI 2024 When is a Metaphor Actually Novel? Annotating Metaphor Novelty in the Context of Automatic Metaphor Detection EACL 2024 When Reasoning Meets Information Aggregation: A Case Study with Sports Narratives EMNLP 2024 Who Evaluates the Evaluations? Objectively Scoring Text-to-Image Prompt Coherence Metrics with T2IScoreScore (TS2) NeurIPS 2024 Why Don’t Prompt-Based Fairness Metrics Correlate? ACL 2024 WMT24 Test Suite: Gender Resolution in Speaker-Listener Dialogue Roles EMNLP 2024 WONDERBREAD: A Benchmark for Evaluating Multimodal Foundation Models on Business Process Management Tasks NeurIPS 2024 Zipper: Addressing Degeneracy in Algorithm-Agnostic Inference NeurIPS 2024