Adina Williams

Calibrating LLM Judges: Linear Probes for Fast and Reliable Uncertainty Estimation ACL 2026 Arbiters of Ambivalence: Challenges of using LLMs in No-Consensus tasks ACL 2025 Domain Regeneration: How well do LLMs match syntactic properties of text domains? ACL 2025 Findings of the Third BabyLM Challenge: Accelerating Language Modeling Research with Cognitively Plausible Data EMNLP 2025 Improving Model Evaluation using SMART Filtering of Benchmark Datasets NAACL 2025 On the Role of Speech Data in Reducing Toxicity Detection Bias NAACL 2025 Are Female Carpenters like Blue Bananas? A Corpus Investigation of Occupation Gender Typicality ACL 2024 EmphAssess : a Prosodic Benchmark on Assessing Emphasis Transfer in Speech-to-Speech Models EMNLP 2024 Findings of the Second BabyLM Challenge: Sample-Efficient Pretraining on Developmentally Plausible Corpora CoNLL 2024 The Factorization Curse: Which Tokens You Predict Underlie the Reversal Curse and More NeurIPS 2024 The PRISM Alignment Dataset: What Participatory, Representative and Individualised Human Feedback Reveals About the Subjective and Multicultural Alignment of Large Language Models NeurIPS 2024 A Latent-Variable Model for Intrinsic Probing AAAI 2023 DataPerf: Benchmarks for Data-Centric AI Development NeurIPS 2023 Findings of the BabyLM Challenge: Sample-Efficient Pretraining on Developmentally Plausible Corpora CoNLL 2023 Language model acceptability judgements are not always robust to context ACL 2023 ROBBIE: Robust Bias Evaluation of Large Generative Language Models EMNLP 2023 Robustness of Named-Entity Replacements for In-Context Learning EMNLP 2023 The Gender-GAP Pipeline: A Gender-Aware Polyglot Pipeline for Gender Characterisation in 55 Languages EMNLP 2023 The Validity of Evaluation Results: Assessing Concurrence Across Compositionality Benchmarks CoNLL 2023 Analyzing Dynamic Adversarial Training Data in the Limit ACL 2022 Benchmarking Compositionality with Formal Languages COLING 2022 Dynatask: A Framework for Creating Dynamic AI Benchmark Tasks ACL 2022 “I’m sorry to hear that”: Finding New Biases in Language Models with a Holistic Descriptor Dataset EMNLP 2022 Investigating Failures of Automatic Translationin the Case of Unambiguous Gender ACL 2022 On the Machine Learning of Ethical Judgments from Natural Language NAACL 2022 Perturbation Augmentation for Fairer NLP EMNLP 2022 The Curious Case of Absolute Position Embeddings EMNLP 2022 Winoground: Probing Vision and Language Models for Visio-Linguistic Compositionality CVPR 2022 Dynabench: Rethinking Benchmarking in NLP NAACL 2021 Dynaboard: An Evaluation-As-A-Service Platform for Holistic Next-Generation Benchmarking NeurIPS 2021 Generalising to German Plural Noun Classes, from the Perspective of a Recurrent Neural Network CoNLL 2021 Masked Language Modeling and the Distributional Hypothesis: Order Word Matters Pre-training for Little EMNLP 2021 Sometimes We Want Ungrammatical Translations EMNLP 2021 To what extent do human explanations of model behavior align with actual model behavior? EMNLP 2021 UnNatural Language Inference ACL 2021 Adversarial NLI: A New Benchmark for Natural Language Understanding ACL 2020 Are Natural Language Inference Models IMPPRESsive? Learning IMPlicature and PRESupposition ACL 2020 A Tale of a Probe and a Parser ACL 2020 Information-Theoretic Probing for Linguistic Structure ACL 2020 Intrinsic Probing through Dimension Selection EMNLP 2020 Measuring the Similarity of Grammatical Gender Systems by Comparing Partitions EMNLP 2020 Multi-Dimensional Gender Bias Classification EMNLP 2020 Pareto Probing: Trading Off Accuracy for Complexity EMNLP 2020 Predicting Declension Class from Form and Meaning ACL 2020 Queens are Powerful too: Mitigating Gender Bias in Dialogue Generation EMNLP 2020 SIGMORPHON 2020 Shared Task 0: Typologically Diverse Morphological Inflection ACL 2020 On the Idiosyncrasies of the Mandarin Chinese Classifier System NAACL 2019 Quantifying the Semantic Core of Gender Systems EMNLP 2019 A Broad-Coverage Challenge Corpus for Sentence Understanding through Inference NAACL 2018 XNLI: Evaluating Cross-lingual Sentence Representations EMNLP 2018