language model pretraining

PMI-Masking: Principled masking of correlated spans ICLR 2021 Pre-training Methods for Neural Machine Translation ACL 2021 Pre-train or Annotate? Domain Adaptation with a Constrained Budget EMNLP 2021 T-NER: An All-Round Python Library for Transformer-based Named Entity Recognition EACL 2021 ZJUKLAB at SemEval-2021 Task 4: Negative Augmentation with Language Model for Reading Comprehension of Abstract Meaning SemEval 2021 An Empirical Investigation Towards Efficient Multi-Domain Language Model Pre-training EMNLP 2020 Byte Pair Encoding is Suboptimal for Language Model Pretraining EMNLP 2020 Investigating Learning Dynamics of BERT Fine-Tuning AACL 2020 Large Product Key Memory for Pretrained Language Models EMNLP 2020 Learning Spoken Language Representations with Neural Lattice Language Modeling ACL 2020 Let’s Stop Incorrect Comparisons in End-to-end Relation Extraction! EMNLP 2020 PALM: Pre-training an Autoencoding&Autoregressive Language Model for Context-conditioned Generation EMNLP 2020 Pretrained Language Model Embryology: The Birth of ALBERT EMNLP 2020 Rare Words: A Major Problem for Contextualized Embeddings and How to Fix it by Attentive Mimicking AAAI 2020 Retrieval Augmented Language Model Pre-Training ICML 2020 Reusing a Pretrained Language Model on Languages with Limited Corpora for Unsupervised NMT EMNLP 2020 Revisiting Pre-Trained Models for Chinese Natural Language Processing EMNLP 2020 Self-Supervised Meta-Learning for Few-Shot Natural Language Classification Tasks EMNLP 2020 SenseBERT: Driving Some Sense into BERT ACL 2020 SJTU-NICT’s Supervised and Unsupervised Neural Machine Translation Systems for the WMT20 News Translation Task EMNLP 2020 TweetEval: Unified Benchmark and Comparative Evaluation for Tweet Classification EMNLP 2020 Cloze-driven Pretraining of Self-attention Networks EMNLP 2019 Cross-Lingual Training for Automatic Question Generation ACL 2019 English-Myanmar Supervised and Unsupervised NMT: NICT’s Machine Translation Systems at WAT-2019 EMNLP 2019 The University of Edinburgh’s Submissions to the WMT19 News Translation Task ACL 2019