Evaluation

Fine-Tuned Machine Translation Metrics Struggle in Unseen Domains ACL 2024 FlashEval: Towards Fast and Accurate Evaluation of Text-to-image Diffusion Generative Models CVPR 2024 FlowBench: Revisiting and Benchmarking Workflow-Guided Planning for LLM-based Agents EMNLP 2024 FOFO: A Benchmark to Evaluate LLMs’ Format-Following Capability ACL 2024 FOOL ME IF YOU CAN! An Adversarial Dataset to Investigate the Robustness of LMs in Word Sense Disambiguation EMNLP 2024 Forgetting Curve: A Reliable Method for Evaluating Memorization Capability for Long-Context Models EMNLP 2024 Foundational Autoraters: Taming Large Language Models for Better Automatic Evaluation EMNLP 2024 Fréchet Distance for Offline Evaluation of Information Retrieval Systems with Sparse Labels EACL 2024 FRoG: Evaluating Fuzzy Reasoning of Generalized Quantifiers in LLMs EMNLP 2024 From Generation to Selection: Findings of Converting Analogical Problem-Solving into Multiple-Choice Questions EMNLP 2024 From Model-centered to Human-Centered: Revision Distance as a Metric for Text Evaluation in LLMs-based Applications ACL 2024 From Test-Taking to Test-Making: Examining LLM Authoring of Commonsense Assessment Items EMNLP 2024 Full Bayesian Significance Testing for Neural Networks AAAI 2024 Fundamental Capabilities of Large Language Models and their Applications in Domain Scenarios: A Survey ACL 2024 GenAI Arena: An Open Evaluation Platform for Generative Models NeurIPS 2024 Generalization or Memorization: Data Contamination and Trustworthy Evaluation for Large Language Models ACL 2024 Generative Interpretation: Toward Human-Like Evaluation for Educational Question-Answer Pair Generation EACL 2024 GPT is Not an Annotator: The Necessity of Human Annotation in Fairness Benchmark Construction ACL 2024 Greed is All You Need: An Evaluation of Tokenizer Inference Methods ACL 2024 GSM-Plus: A Comprehensive Benchmark for Evaluating the Robustness of LLMs as Mathematical Problem Solvers ACL 2024 Guardians of the Machine Translation Meta-Evaluation: Sentinel Metrics Fall In! ACL 2024 HANS, are you clever? Clever Hans Effect Analysis of Neural Systems NAACL 2024 HelloFresh: LLM Evalutions on Streams of Real-World Human Editorial Actions across X Community Notes and Wikipedia edits ACL 2024 Holistic Evaluation of Large Language Models: Assessing Robustness, Accuracy, and Toxicity for Real-World Applications NAACL 2024 How Hard is this Test Set? NLI Characterization by Exploiting Training Dynamics EMNLP 2024