language model evaluation

221 papers

Explore in graph

Also known as

LLM EVALUATION LME LLM LMS

Co-occurring keywords

large language model (12755) language model (4573) benchmark evaluation (1539) multilingual nlp (1423) natural language understanding (845) text generation (2903) low-resource language (2234) evaluation benchmark (250) text classification (6776) question answering (2904)

Papers

JCoLA: Japanese Corpus of Linguistic Acceptability COLING 2024

Prometheus 2: An Open Source Language Model Specialized in Evaluating Other Language Models EMNLP 2024

Token-length Bias in Minimal-pair Paradigm Datasets COLING 2024

Long-form factuality in large language models NIPS 2024

ArabicMMLU: Assessing Massive Multitask Language Understanding in Arabic ACL 2024

FRoG: Evaluating Fuzzy Reasoning of Generalized Quantifiers in LLMs EMNLP 2024

RuBLiMP: Russian Benchmark of Linguistic Minimal Pairs EMNLP 2024

“A good pun is its own reword”: Can Large Language Models Understand Puns? EMNLP 2024

ReadMe++: Benchmarking Multilingual Language Models for Multi-Domain Readability Assessment EMNLP 2024

Finding Blind Spots in Evaluator LLMs with Interpretable Checklists EMNLP 2024

Noot Noot at SemEval-2024 Task 7: Numerical Reasoning and Headline Generation SEMEVAL 2024

ModeLing: A Novel Dataset for Testing Linguistic Reasoning in Language Models EACL 2024

Stratified Prediction-Powered Inference for Effective Hybrid Evaluation of Language Models NIPS 2024

UltraEval: A Lightweight Platform for Flexible and Comprehensive Evaluation for LLMs ACL 2024

Elo Uncovered: Robustness and Best Practices in Language Model Evaluation NIPS 2024

Can Multiple-choice Questions Really Be Useful in Detecting the Abilities of LLMs? COLING 2024

Factcheck-Bench: Fine-Grained Evaluation Benchmark for Automatic Fact-checkers EMNLP 2024

SimLex-999 for Dutch COLING 2024

Efficient Benchmarking (of Language Models) NAACL 2024

PatentEval: Understanding Errors in Patent Generation NAACL 2024

TofuEval: Evaluating Hallucinations of LLMs on Topic-Focused Dialogue Summarization NAACL 2024

Susu Box or Piggy Bank: Assessing Cultural Commonsense Knowledge between Ghana and the US EMNLP 2024

RepLiQA: A Question-Answering Dataset for Benchmarking LLMs on Unseen Reference Content NIPS 2024

MT-Eval: A Multi-Turn Capabilities Evaluation Benchmark for Large Language Models EMNLP 2024

BertaQA: How Much Do Language Models Know About Local Culture? NIPS 2024