Evaluation

INVITE: a Testbed of Automatically Generated Invalid Questions to Evaluate Large Language Models for Hallucinations EMNLP 2023 In What Languages are Generative Language Models the Most Formal? Analyzing Formality Distribution across Languages EMNLP 2023 Is GPT-4 a Good Data Analyst? EMNLP 2023 Iterative Approximate Cross-Validation ICML 2023 It is a Bird Therefore it is a Robin: On BERT’s Internal Consistency Between Hypernym Knowledge and Logical Words ACL 2023 Judging LLM-as-a-Judge with MT-Bench and Chatbot Arena NeurIPS 2023 Large Language Models are Not Yet Human-Level Evaluators for Abstractive Summarization EMNLP 2023 Large Language Models respond to Influence like Humans ACL 2023 Larger Probes Tell a Different Story: Extending Psycholinguistic Datasets Via In-Context Learning EMNLP 2023 Legal Judgment Prediction: If You Are Going to Do It, Do It Right EMNLP 2023 Lenient Evaluation of Japanese Speech Recognition: Modeling Naturally Occurring Spelling Inconsistency ACL 2023 Lexical Repetitions Lead to Rote Learning: Unveiling the Impact of Lexical Overlap in Train and Test Reference Summaries EMNLP 2023 LEXTREME: A Multi-Lingual and Multi-Task Benchmark for the Legal Domain EMNLP 2023 LLM-Eval: Unified Multi-Dimensional Automatic Evaluation for Open-Domain Conversations with Large Language Models ACL 2023 LMentry: A Language Model Benchmark of Elementary Language Tasks ACL 2023 Mass-Producing Failures of Multimodal Systems with Language Models NeurIPS 2023 Mathematical Capabilities of ChatGPT NeurIPS 2023 Measuring Consistency in Text-based Financial Forecasting Models ACL 2023 Measuring Pointwise 𝒱-Usable Information In-Context-ly EMNLP 2023 Measuring the Instability of Fine-Tuning ACL 2023 Measuring the Knowledge Acquisition-Utilization Gap in Pretrained Language Models EMNLP 2023 Mind the Gap between the Application Track and the Real World ACL 2023 MISMATCH: Fine-grained Evaluation of Machine-generated Text with Mismatch Error Types ACL 2023 MM-SHAP: A Performance-agnostic Metric for Measuring Multimodal Contributions in Vision and Language Models & Tasks ACL 2023 MOCKS 1.0: Multilingual Open Custom Keyword Spotting Testset Interspeech 2023