conftrace_

2021 EMNLP EMNLP 2021

Are BERTs Sensitive to Native Interference in L2 Production?

Abstract

AbstractWith the essays part from The International Corpus Network of Asian Learners of English (ICNALE) and the TOEFL11 corpus, we fine-tuned neural language models based on BERT to predict English learners’ native languages. Results showed neural models can learn to represent and detect such native language impacts, but multilingually trained models have no advantage in doing so.

❓ The Questioner

🌉 Interdisciplinary Bridge - Artificial Intelligence and Natural Language Processing

Authors

Zixin Tang, Prasenjit Mitra, David Reitter

Topics

Deep Learning > Architectures > Transformers Natural Language Processing > Applications > Text Classification Natural Language Processing > Resources & Methods > Large Language Models Natural Language Processing > Resources & Methods > Multilingual NLP Machine Learning > Learning Types > Transfer Learning Deep Learning > Learning Types > Fine-Tuning Natural Language Processing > Applications > Natural Language Understanding

Keywords

transfer learning language model multilingual model native language neural language model second language acquisition native language interference

More from EMNLP 2021

All EMNLP 2021 papers →

BERT Cannot Align Characters

Two Heads are Better than One? Verification of Ensemble Effect in Neural Machine Translation

Finetuning Pretrained Transformers into Variational Autoencoders

Zero-Shot Cross-Lingual Transfer is a Hard Baseline to Beat in German Fine-Grained Entity Typing

Comparing Euclidean and Hyperbolic Embeddings on the WordNet Nouns Hypernymy Graph

When does Further Pre-training MLM Help? An Empirical Study on Task-Oriented Dialog Pre-training