benchmark evaluation

GuardBench: A Large-Scale Benchmark for Guardrail Models EMNLP 2024 HallusionBench: An Advanced Diagnostic Suite for Entangled Language Hallucination and Visual Illusion in Large Vision-Language Models CVPR 2024 Head-to-Tail: How Knowledgeable are Large Language Models (LLMs)? A.K.A. Will LLMs Replace Knowledge Graphs? NAACL 2024 HelloFresh: LLM Evalutions on Streams of Real-World Human Editorial Actions across X Community Notes and Wikipedia edits ACL 2024 How to Evaluate the Generalization of Detection? A Benchmark for Comprehensive Open-Vocabulary Detection AAAI 2024 I2EBench: A Comprehensive Benchmark for Instruction-based Image Editing NeurIPS 2024 I Could’ve Asked That: Reformulating Unanswerable Questions EMNLP 2024 II-Bench: An Image Implication Understanding Benchmark for Multimodal Large Language Models NeurIPS 2024 ImplicitAVE: An Open-Source Dataset and Multimodal LLMs Benchmark for Implicit Attribute Value Extraction ACL 2024 InfiBench: Evaluating the Question-Answering Capabilities of Code Large Language Models NeurIPS 2024 InFoBench: Evaluating Instruction Following Ability in Large Language Models ACL 2024 InstructEval: Systematic Evaluation of Instruction Selection Methods NAACL 2024 Instruction Fine-Tuning: Does Prompt Loss Matter? EMNLP 2024 IntentionQA: A Benchmark for Evaluating Purchase Intention Comprehension Abilities of Language Models in E-commerce EMNLP 2024 INTERSPEECH 2009 Emotion Challenge Revisited: Benchmarking 15 Years of Progress in Speech Emotion Recognition Interspeech 2024 Investigating Data Contamination in Modern Benchmarks for Large Language Models NAACL 2024 Is A Picture Worth A Thousand Words? Delving Into Spatial Reasoning for Vision Language Models NeurIPS 2024 JailbreakBench: An Open Robustness Benchmark for Jailbreaking Large Language Models NeurIPS 2024 Jargon: A Suite of Language Models and Evaluation Tasks for French Specialized Domains COLING 2024 JourneyBench: A Challenging One-Stop Vision-Language Understanding Benchmark of Generated Images NeurIPS 2024 KoDialogBench: Evaluating Conversational Understanding of Language Models with Korean Dialogue Benchmark COLING 2024 KoLA: Carefully Benchmarking World Knowledge of Large Language Models ICLR 2024 Large Language Models Are Not Strong Abstract Reasoners IJCAI 2024 Large Language Models Are Poor Clinical Decision-Makers: A Comprehensive Benchmark EMNLP 2024 Large Language Models Can Not Perform Well in Understanding and Manipulating Natural Language at Both Character and Word Levels? EMNLP 2024