Evaluation

Quality Matters: Evaluating Synthetic Data for Tool-Using LLMs EMNLP 2024 Quantifying Contamination in Evaluating Code Generation Capabilities of Language Models ACL 2024 Quantifying Generalizations: Exploring the Divide Between Human and LLMs’ Sensitivity to Quantification ACL 2024 Questioning the Survey Responses of Large Language Models NeurIPS 2024 RAGTruth: A Hallucination Corpus for Developing Trustworthy Retrieval-Augmented Language Models ACL 2024 Ranking Large Language Models without Ground Truth ACL 2024 Rationale-Aware Answer Verification by Pairwise Self-Evaluation EMNLP 2024 Rationales for Answers to Simple Math Word Problems Confuse Large Language Models ACL 2024 Reasoning or a Semblance of it? A Diagnostic Study of Transitive Reasoning in LLMs EMNLP 2024 ReCaLL: Membership Inference via Relative Conditional Log-Likelihoods EMNLP 2024 Re-Evaluating Evaluation for Multilingual Summarization EMNLP 2024 Reference-based Metrics Disprove Themselves in Question Generation EMNLP 2024 Reframing Spatial Reasoning Evaluation in Language Models: A Real-World Simulation Benchmark for Qualitative Reasoning IJCAI 2024 Remember This Event That Year? Assessing Temporal Information and Understanding in Large Language Models EMNLP 2024 ReMI: A Dataset for Reasoning with Multiple Images NeurIPS 2024 RepEval: Effective Text Evaluation with LLM Representation EMNLP 2024 Replicability in Learning: Geometric Partitions and KKM-Sperner Lemma NeurIPS 2024 RepLiQA: A Question-Answering Dataset for Benchmarking LLMs on Unseen Reference Content NeurIPS 2024 RepMatch: Quantifying Cross-Instance Similarities in Representation Space EMNLP 2024 Reproduce, Replicate, Reevaluate. The Long but Safe Way to Extend Machine Learning Methods AAAI 2024 Reshuffling Resampling Splits Can Improve Generalization of Hyperparameter Optimization NeurIPS 2024 Rethinking Evaluation Methods for Machine Unlearning EMNLP 2024 Rethinking FID: Towards a Better Evaluation Metric for Image Generation CVPR 2024 Rethinking LLM Memorization through the Lens of Adversarial Compression NeurIPS 2024 Rethinking the Evaluation Protocol of Domain Generalization CVPR 2024