model comparison

PandaLM: An Automatic Evaluation Benchmark for LLM Instruction Tuning Optimization ICLR 2024 Perturbed examples reveal invariances shared by language models ACL 2024 RepMatch: Quantifying Cross-Instance Similarities in Representation Space EMNLP 2024 A Confederacy of Models: a Comprehensive Evaluation of LLMs on Creative Writing EMNLP 2023 Do Question Answering Modeling Improvements Hold Across Benchmarks? ACL 2023 Follow the leader(board) with confidence: Estimating p-values from a single test set with item and response variance ACL 2023 Holistic Evaluation of Text-to-Image Models NeurIPS 2023 ModelDiff: A Framework for Comparing Learning Algorithms ICML 2023 On “Scientific Debt” in NLP: A Case for More Rigour in Language Model Pre-Training Research ACL 2023 The Devil is in the Details: On the Pitfalls of Event Extraction Evaluation ACL 2023 We Need to Talk About Reproducibility in NLP Model Comparison EMNLP 2023 Evaluation Metrics for Graph Generative Models: Problems, Pitfalls, and Practical Solutions ICLR 2022 HAPI: A Large-scale Longitudinal Dataset of Commercial ML API Predictions NeurIPS 2022 Near-Negative Distinction: Giving a Second Life to Human Evaluation Datasets EMNLP 2022 Pythae: Unifying Generative Autoencoders in Python - A Benchmarking Use Case NeurIPS 2022 Trivial or Impossible --- dichotomous data difficulty masks model differences (on ImageNet and beyond) ICLR 2022 Are Pretrained Convolutions Better than Pretrained Transformers? ACL 2021 Comparing Symbolic Models of Language via Bayesian Inference (Student Abstract) AAAI 2021 Dynaboard: An Evaluation-As-A-Service Platform for Holistic Next-Generation Benchmarking NeurIPS 2021 Team 9: A Comparison of Simple vs. Complex Models for Suicide Risk Assessment NAACL 2021 What Changed? Interpretable Model Comparison IJCAI 2021 Beyond accuracy: quantifying trial-by-trial behaviour of CNNs and humans by measuring error consistency NeurIPS 2020 Let’s Stop Incorrect Comparisons in End-to-end Relation Extraction! EMNLP 2020 Comparing Sample-Wise Learnability across Deep Neural Network Models AAAI 2019 Deep Dominance - How to Properly Compare Deep Neural Models ACL 2019