benchmark evaluation

LatestEval: Addressing Data Contamination in Language Model Evaluation through Dynamic and Time-Sensitive Test Construction AAAI 2024 LawBench: Benchmarking Legal Knowledge of Large Language Models EMNLP 2024 LEGOBench: Scientific Leaderboard Generation Benchmark EMNLP 2024 LexEval: A Comprehensive Chinese Legal Benchmark for Evaluating Large Language Models NeurIPS 2024 LHMKE: A Large-scale Holistic Multi-subject Knowledge Evaluation Benchmark for Chinese Large Language Models COLING 2024 Liar, Liar, Logical Mire: A Benchmark for Suppositional Reasoning in Large Language Models EMNLP 2024 Living in the Moment: Can Large Language Models Grasp Co-Temporal Reasoning? ACL 2024 LLMArena: Assessing Capabilities of Large Language Models in Dynamic Multi-Agent Environments ACL 2024 LLMCBench: Benchmarking Large Language Model Compression for Efficient Deployment NeurIPS 2024 LLMeBench: A Flexible Framework for Accelerating LLMs Benchmarking EACL 2024 LLM-Evolve: Evaluation for LLM’s Evolving Capability on Benchmarks EMNLP 2024 LogicBench: Towards Systematic Evaluation of Logical Reasoning Ability of Large Language Models ACL 2024 LongBench: A Bilingual, Multitask Benchmark for Long Context Understanding ACL 2024 LongGenBench: Long-context Generation Benchmark EMNLP 2024 LooGLE: Can Long-Context Language Models Understand Long Contexts? ACL 2024 Losing Visual Needles in Image Haystacks: Vision Language Models are Easily Distracted in Short and Long Contexts EMNLP 2024 LOVA3: Learning to Visual Question Answering, Asking and Assessment NeurIPS 2024 M3CoT: A Novel Benchmark for Multi-Domain Multi-step Multi-modal Chain-of-Thought ACL 2024 M4GT-Bench: Evaluation Benchmark for Black-Box Machine-Generated Text Detection ACL 2024 M4LE: A Multi-Ability Multi-Range Multi-Task Multi-Domain Long-Context Evaluation Benchmark for Large Language Models ACL 2024 M5 – A Diverse Benchmark to Assess the Performance of Large Multimodal Models Across Multilingual and Multicultural Vision-Language Tasks EMNLP 2024 MAFALDA: A Benchmark and Comprehensive Study of Fallacy Detection and Classification NAACL 2024 MAgIC: Investigation of Large Language Model Powered Multi-Agent in Cognition, Adaptability, Rationality and Collaboration EMNLP 2024 MAIR: A Massive Benchmark for Evaluating Instructed Retrieval EMNLP 2024 MAmmoTH2: Scaling Instructions from the Web NeurIPS 2024