Evaluation

Scaling Laws of Synthetic Images for Model Training ... for Now CVPR 2024 SceMQA: A Scientific College Entrance Level Multimodal Question Answering Benchmark ACL 2024 Self-Evaluation of Large Language Model based on Glass-box Features EMNLP 2024 Social Intelligence Data Infrastructure: Structuring the Present and Navigating the Future ACL 2024 Split and Merge: Aligning Position Biases in LLM-based Evaluators EMNLP 2024 StructEval: Deepen and Broaden Large Language Model Assessment via Structured Evaluation ACL 2024 SUPER: Evaluating Agents on Setting Up and Executing Tasks from Research Repositories EMNLP 2024 Synthesize Diagnose and Optimize: Towards Fine-Grained Vision-Language Understanding CVPR 2024 The Base-Rate Effect on LLM Benchmark Performance: Disambiguating Test-Taking Strategies from Benchmark Performance EMNLP 2024 The Butterfly Effect of Model Editing: Few Edits Can Trigger Large Language Models Collapse ACL 2024 The Effect of Sampling Temperature on Problem Solving in Large Language Models EMNLP 2024 The Instinctive Bias: Spurious Images lead to Illusion in MLLMs EMNLP 2024 The Music Maestro or The Musically Challenged, A Massive Music Evaluation Benchmark for Large Language Models ACL 2024 ToolBeHonest: A Multi-level Hallucination Diagnostic Benchmark for Tool-Augmented Large Language Models EMNLP 2024 Towards Benchmarking Situational Awareness of Large Language Models:Comprehensive Benchmark, Evaluation and Analysis EMNLP 2024 Towards Reproducible, Automated, and Scalable Anomaly Detection AAAI 2024 TOWER: Tree Organized Weighting for Evaluating Complex Instructions EMNLP 2024 TuringQ: Benchmarking AI Comprehension in Theory of Computation EMNLP 2024 Uncovering Limitations of Large Language Models in Information Seeking from Tables ACL 2024 Unveiling the Bias Impact on Symmetric Moral Consistency of Large Language Models NeurIPS 2024 VarBench: Robust Language Model Benchmarking Through Dynamic Variable Perturbation EMNLP 2024 VBench: Comprehensive Benchmark Suite for Video Generative Models CVPR 2024 VideoScore: Building Automatic Metrics to Simulate Fine-grained Human Feedback for Video Generation EMNLP 2024 VLEU: a Method for Automatic Evaluation for Generalizability of Text-to-Image Models EMNLP 2024 Weak-eval-Strong: Evaluating and Eliciting Lateral Thinking of LLMs with Situation Puzzles NeurIPS 2024