language model evaluation

MedEval: A Multi-Level, Multi-Task, and Multi-Domain Medical Benchmark for Language Model Evaluation EMNLP 2023 MoCa: Measuring Human-Language Model Alignment on Causal and Moral Judgment Tasks NeurIPS 2023 OpenICL: An Open-Source Framework for In-context Learning ACL 2023 ORCA: A Challenging Benchmark for Arabic Language Understanding ACL 2023 Pseudointelligence: A Unifying Lens on Language Model Evaluation EMNLP 2023 Retrieval-based Evaluation for LLMs: A Case Study in Korean Legal QA EMNLP 2023 Selective-LAMA: Selective Prediction for Confidence-Aware Evaluation of Language Models EACL 2023 SuperTweetEval: A Challenging, Unified and Heterogeneous Benchmark for Social Media NLP Research EMNLP 2023 Transparency at the Source: Evaluating and Interpreting Language Models With Access to the True Distribution EMNLP 2023 UINAUIL: A Unified Benchmark for Italian Natural Language Understanding ACL 2023 A Dog Is Passing Over The Jet? A Text-Generation Dataset for Korean Commonsense Reasoning and Evaluation NAACL 2022 A Knowledge-Graph-Based Intrinsic Test for Benchmarking Medical Concept Embeddings and Pretrained Language Models EMNLP 2022 BECEL: Benchmark for Consistency Evaluation of Language Models COLING 2022 BERTScore is Unfair: On Social Bias in Language Model-Based Metrics for Text Generation EMNLP 2022 Beyond Static models and test sets: Benchmarking the potential of pre-trained models across tasks and languages ACL 2022 ChapterBreak: A Challenge Dataset for Long-Range Language Models NAACL 2022 Characteristics of Harmful Text: Towards Rigorous Benchmarking of Language Models NeurIPS 2022 Curriculum: A Broad-Coverage Benchmark for Linguistic Phenomena in Natural Language Understanding NAACL 2022 DiscoSense: Commonsense Reasoning with Discourse Connectives EMNLP 2022 GeoMLAMA: Geo-Diverse Commonsense Probing on Multilingual Pre-Trained Language Models EMNLP 2022 GiCCS: A German in-Context Conversational Similarity Benchmark EMNLP 2022 How Conservative are Language Models? Adapting to the Introduction of Gender-Neutral Pronouns NAACL 2022 Language Models Are Poor Learners of Directional Inference EMNLP 2022 Numerical Correlation in Text EMNLP 2022 On the Cusp of Comprehensibility: Can Language Models Distinguish Between Metaphors and Nonsense? EMNLP 2022