Skip to content
conftrace_
2021 EMNLP EMNLP 2021

Generalization in NLI: Ways (Not) To Go Beyond Simple Heuristics

Abstract

AbstractMuch of recent progress in NLU was shown to be due to models’ learning dataset-specific heuristics. We conduct a case study of generalization in NLI (from MNLI to the adversarially constructed HANS dataset) in a range of BERT-based architectures (adapters, Siamese Transformers, HEX debiasing), as well as with subsampling the data and increasing the model size. We report 2 successful and 3 unsuccessful strategies, all providing insights into how Transformer-based models learn to generalize.

🌉 Interdisciplinary Bridge - Artificial Intelligence and Natural Language Processing

Authors

Prajjwal Bhargava, Aleksandr Drozd, Anna Rogers

Topics

Artificial Intelligence > Core AI > Interpretability Machine Learning > Learning Types > Adversarial Learning Machine Learning > Learning Types > Weakly Supervised Learning Deep Learning > Architectures > Transformers Natural Language Processing > Applications > Text Classification Natural Language Processing > Applications > Natural Language Inference Machine Learning > Learning Types > Deep Learning Machine Learning > Optimization & Theory > Generalization Deep Learning > Optimization & Theory > Generalization

Keywords

text classification natural language inference model generalization adversarial dataset adversarial evaluation adversarial generalization dataset-specific heuristic
Download PDF

More from EMNLP 2021

All EMNLP 2021 papers →
An Investigation into the Contribution of Locally Aggregated Descriptors to Figurative Language Identification
Blindness to Modality Helps Entailment Graph Mining
Investigating the Effect of Natural Language Explanations on Out-of-Distribution Generalization in Few-shot NLI
Challenging the Semi-Supervised VAE Framework for Text Classification
Active Learning for Argument Strength Estimation
The Early Modern Dutch Mediascape. Detecting Media Mentions in Chronicles Using Word Embeddings and CRF