masked language modeling

Segatron: Segment-Aware Transformer for Language Modeling and Understanding AAAI 2021 Self-Supervised Quality Estimation for Machine Translation EMNLP 2021 Sentence Bottleneck Autoencoders from Transformer Language Models EMNLP 2021 SJ_AJ@DravidianLangTech-EACL2021: Task-Adaptive Pre-Training of Multilingual BERT models for Offensive Language Identification EACL 2021 SPLAT: Speech-Language Joint Pre-Training for Spoken Language Understanding NAACL 2021 StructFormer: Joint Unsupervised Induction of Dependency and Constituency Structure from Masked Language Modeling ACL 2021 SupCL-Seq: Supervised Contrastive Learning for Downstream Optimized Sequence Representations EMNLP 2021 TAPAS at SemEval-2021 Task 9: Reasoning over tables with intermediate pre-training SemEval 2021 TAP: Text-Aware Pre-Training for Text-VQA and Text-Caption CVPR 2021 Target-Aware Data Augmentation for Stance Detection NAACL 2021 Temporal Adaptation of BERT and Performance on Downstream Document Classification: Insights from Social Media EMNLP 2021 The Effectiveness of Intermediate-Task Training for Code-Switched Natural Language Understanding EMNLP 2021 Towards Semantics-Enhanced Pre-Training: Can Lexicon Definitions Help Learning Sentence Meanings? AAAI 2021 UC2: Universal Cross-Lingual Cross-Modal Vision-and-Language Pre-Training CVPR 2021 Universal Sentence Representation Learning with Conditional Masked Language Model EMNLP 2021 Unsupervised Pronoun Resolution via Masked Noun-Phrase Prediction ACL 2021 Vision-and-Language or Vision-for-Language? On Cross-Modal Influence in Multimodal Transformers EMNLP 2021 When does Further Pre-training MLM Help? An Empirical Study on Task-Oriented Dialog Pre-training EMNLP 2021 Birds have four legs?! NumerSense: Probing Numerical Commonsense Knowledge of Pre-Trained Language Models EMNLP 2020 CoLAKE: Contextualized Language and Knowledge Embedding COLING 2020 Controlling the Imprint of Passivization and Negation in Contextualized Representations EMNLP 2020 Cross-Thought for Sentence Encoder Pre-training EMNLP 2020 Deep Transformers with Latent Depth NeurIPS 2020 ELECTRA: Pre-training Text Encoders as Discriminators Rather Than Generators ICLR 2020 Emerging Cross-lingual Structure in Pretrained Language Models ACL 2020