Evaluation

How Reliable Are Automatic Evaluation Methods for Instruction-Tuned LLMs? EMNLP 2024 How to Evaluate the Generalization of Detection? A Benchmark for Comprehensive Open-Vocabulary Detection AAAI 2024 Humans or LLMs as the Judge? A Study on Judgement Bias EMNLP 2024 Hypothesis Testing for Class-Conditional Noise Using Local Maximum Likelihood AAAI 2024 I2EBench: A Comprehensive Benchmark for Instruction-based Image Editing NeurIPS 2024 I am a Strange Dataset: Metalinguistic Tests for Language Models ACL 2024 I Could’ve Asked That: Reformulating Unanswerable Questions EMNLP 2024 IDGen: Item Discrimination Induced Prompt Generation for LLM Evaluation NeurIPS 2024 ImageNet-D: Benchmarking Neural Network Robustness on Diffusion Synthetic Object CVPR 2024 Improving GNN Calibration with Discriminative Ability: Insights and Strategies AAAI 2024 Improving Statistical Significance in Human Evaluation of Automatic Metrics via Soft Pairwise Accuracy EMNLP 2024 In2Core: Leveraging Influence Functions for Coreset Selection in Instruction Finetuning of Large Language Models EMNLP 2024 Information-theoretic Generalization Analysis for Expected Calibration Error NeurIPS 2024 In Search of the Long-Tail: Systematic Generation of Long-Tail Inferential Knowledge via Logical Rule Guided Search EMNLP 2024 InstructEval: Towards Holistic Evaluation of Instruction-Tuned Large Language Models EACL 2024 IntentionQA: A Benchmark for Evaluating Purchase Intention Comprehension Abilities of Language Models in E-commerce EMNLP 2024 InterIntent: Investigating Social Intelligence of LLMs via Intention Understanding in an Interactive Game Context EMNLP 2024 InterrogateLLM: Zero-Resource Hallucination Detection in LLM-Generated Answers ACL 2024 Introducing GenCeption for Multimodal LLM Benchmarking: You May Bypass Annotations NAACL 2024 Intuitive or Dependent? Investigating LLMs’ Behavior Style to Conflicting Prompts ACL 2024 Is artificial intelligence still intelligence? LLMs generalize to novel adjective-noun pairs, but don’t mimic the full human distribution EMNLP 2024 Is Compound Aspect-Based Sentiment Analysis Addressed by LLMs? EMNLP 2024 Is Cross-validation the Gold Standard to Estimate Out-of-sample Model Performance? NeurIPS 2024 Is It Good Data for Multilingual Instruction Tuning or Just Bad Multilingual Evaluation for Large Language Models? EMNLP 2024 Is LLM a Reliable Reviewer? A Comprehensive Evaluation of LLM on Automatic Paper Reviewing Tasks COLING 2024