Evaluation

LongWanjuan: Towards Systematic Measurement for Long Text Quality EMNLP 2024 LUQ: Long-text Uncertainty Quantification for LLMs EMNLP 2024 M4LE: A Multi-Ability Multi-Range Multi-Task Multi-Domain Long-Context Evaluation Benchmark for Large Language Models ACL 2024 MAIR: A Massive Benchmark for Evaluating Instructed Retrieval EMNLP 2024 Making a Long Story Short in Conversation Modeling EACL 2024 Marathon: A Race Through the Realm of Long Context with Large Language Models ACL 2024 MARiA at SemEval 2024 Task-6: Hallucination Detection Through LLMs, MNLI, and Cosine similarity SemEval 2024 MARS: Meaning-Aware Response Scoring for Uncertainty Estimation in Generative LLMs ACL 2024 MathBench: Evaluating the Theory and Application Proficiency of LLMs with a Hierarchical Mathematics Benchmark ACL 2024 MathFish: Evaluating Language Model Math Reasoning via Grounding in Educational Curricula EMNLP 2024 Measuring Multimodal Mathematical Reasoning with MATH-Vision Dataset NeurIPS 2024 Measuring the Inconsistency of Large Language Models in Preferential Ranking ACL 2024 Measuring the Robustness of NLP Models to Domain Shifts EMNLP 2024 MedCalc-Bench: Evaluating Large Language Models for Medical Calculations NeurIPS 2024 MedJourney: Benchmark and Evaluation of Large Language Models over Patient Clinical Journey NeurIPS 2024 Mercury: A Code Efficiency Benchmark for Code Large Language Models NeurIPS 2024 Meta-Evaluation of Sentence Simplification Metrics COLING 2024 Metrics for What, Metrics for Whom: Assessing Actionability of Bias Evaluation Metrics in NLP EMNLP 2024 MetricX-24: The Google Submission to the WMT 2024 Metrics Shared Task EMNLP 2024 MMLU-Pro: A More Robust and Challenging Multi-Task Language Understanding Benchmark NeurIPS 2024 MM-SAP: A Comprehensive Benchmark for Assessing Self-Awareness of Multimodal Large Language Models in Perception ACL 2024 MMTE: Corpus and Metrics for Evaluating Machine Translation Quality of Metaphorical Language EMNLP 2024 Mobile-Bench: An Evaluation Benchmark for LLM-based Mobile Agents ACL 2024 Moderate Message Passing Improves Calibration: A Universal Way to Mitigate Confidence Bias in Graph Neural Networks AAAI 2024 Monoculture in Matching Markets NeurIPS 2024