Evaluation

CausalBench: A Comprehensive Benchmark for Evaluating Causal Reasoning Capabilities of Large Language Models ACL 2024 Challenging Large Language Models with New Tasks: A Study on their Adaptability and Robustness ACL 2024 CharacterEval: A Chinese Benchmark for Role-Playing Conversational Agent Evaluation ACL 2024 chrF-S: Semantics Is All You Need EMNLP 2024 Classifier Calibration with ROC-Regularized Isotonic Regression AISTATS 2024 CliMedBench: A Large-Scale Chinese Benchmark for Evaluating Medical Large Language Models in Clinical Scenarios EMNLP 2024 CMoralEval: A Moral Evaluation Benchmark for Chinese Large Language Models ACL 2024 Cocktail: A Comprehensive Information Retrieval Benchmark with LLM-Generated Documents Integration ACL 2024 CodeJudge: Evaluating Code Generation with Large Language Models EMNLP 2024 Collaborative Performance Prediction for Large Language Models EMNLP 2024 Compact Proofs of Model Performance via Mechanistic Interpretability NeurIPS 2024 ConceptMath: A Bilingual Concept-wise Benchmark for Measuring Mathematical Reasoning of Large Language Models ACL 2024 Conditional and Modal Reasoning in Large Language Models EMNLP 2024 Confidence Under the Hood: An Investigation into the Confidence-Probability Alignment in Large Language Models ACL 2024 Conformalized Multiple Testing after Data-dependent Selection NeurIPS 2024 Connecting the Dots: Evaluating Abstract Reasoning Capabilities of LLMs Using the New York Times Connections Word Game EMNLP 2024 ConSiDERS-The-Human Evaluation Framework: Rethinking Human Evaluation for Generative Large Language Models ACL 2024 ConStat: Performance-Based Contamination Detection in Large Language Models NeurIPS 2024 Constructing Domain-Specific Evaluation Sets for LLM-as-a-judge EMNLP 2024 Cousins Of The Vendi Score: A Family Of Similarity-Based Diversity Metrics For Science And Machine Learning AISTATS 2024 CriticBench: Benchmarking LLMs for Critique-Correct Reasoning ACL 2024 CriticEval: Evaluating Large-scale Language Model as Critic NeurIPS 2024 CritiqueLLM: Towards an Informative Critique Generation Model for Evaluation of Large Language Model Generation ACL 2024 CryoBench: Diverse and challenging datasets for the heterogeneity problem in cryo-EM NeurIPS 2024 CUTE: Measuring LLMs’ Understanding of Their Tokens EMNLP 2024