language model evaluation

HypoTermQA: Hypothetical Terms Dataset for Benchmarking Hallucination Tendency of LLMs EACL 2024 IndiBias: A Benchmark Dataset to Measure Social Biases in Language Models for Indian Context NAACL 2024 Information Parity: Measuring and Predicting the Multilingual Capabilities of Language Models EMNLP 2024 Is It Good Data for Multilingual Instruction Tuning or Just Bad Multilingual Evaluation for Large Language Models? EMNLP 2024 JCoLA: Japanese Corpus of Linguistic Acceptability COLING 2024 Language Models can Evaluate Themselves via Probability Discrepancy ACL 2024 LatestEval: Addressing Data Contamination in Language Model Evaluation through Dynamic and Time-Sensitive Test Construction AAAI 2024 L-Eval: Instituting Standardized Evaluation for Long Context Language Models ACL 2024 Long-form factuality in large language models NeurIPS 2024 ModeLing: A Novel Dataset for Testing Linguistic Reasoning in Language Models EACL 2024 MT-Eval: A Multi-Turn Capabilities Evaluation Benchmark for Large Language Models EMNLP 2024 “My Answer is C”: First-Token Probabilities Do Not Match Text Answers in Instruction-Tuned Language Models ACL 2024 Noot Noot at SemEval-2024 Task 7: Numerical Reasoning and Headline Generation SemEval 2024 PandaLM: An Automatic Evaluation Benchmark for LLM Instruction Tuning Optimization ICLR 2024 PatentEval: Understanding Errors in Patent Generation NAACL 2024 Prometheus 2: An Open Source Language Model Specialized in Evaluating Other Language Models EMNLP 2024 Prometheus: Inducing Fine-Grained Evaluation Capability in Language Models ICLR 2024 PrOnto: Language Model Evaluations for 859 Languages COLING 2024 Propagation and Pitfalls: Reasoning-based Assessment of Knowledge Editing through Counterfactual Tasks ACL 2024 ReadMe++: Benchmarking Multilingual Language Models for Multi-Domain Readability Assessment EMNLP 2024 RepLiQA: A Question-Answering Dataset for Benchmarking LLMs on Unseen Reference Content NeurIPS 2024 RuBLiMP: Russian Benchmark of Linguistic Minimal Pairs EMNLP 2024 SciDQA: A Deep Reading Comprehension Dataset over Scientific Papers EMNLP 2024 Should We Really Edit Language Models? On the Evaluation of Edited Language Models NeurIPS 2024 SimLex-999 for Dutch COLING 2024