Pretraining

OASIS: A Large-Scale Dataset for Single Image 3D in the Wild CVPR 2020 On the Interplay Between Fine-tuning and Sentence-level Probing for Linguistic Knowledge in Pre-trained Transformers EMNLP 2020 Order Matters: Semantic-Aware Neural Networks for Binary Code Similarity Detection AAAI 2020 PALM: Pre-training an Autoencoding&Autoregressive Language Model for Context-conditioned Generation EMNLP 2020 Parsing as Pretraining AAAI 2020 Parsing with Multilingual BERT, a Small Corpus, and a Small Treebank EMNLP 2020 Phoneme-to-Grapheme Conversion Based Large-Scale Pre-Training for End-to-End Automatic Speech Recognition Interspeech 2020 PLATO: Pre-trained Dialogue Generation Model with Discrete Latent Variable ACL 2020 POINTER: Constrained Progressive Text Generation via Insertion-based Generative Pre-training EMNLP 2020 Poly-encoders: Architectures and Pre-training Strategies for Fast and Accurate Multi-sentence Scoring ICLR 2020 Predicting Responses to Psychological Questionnaires from Participants’ Social Media Posts and Question Text Embeddings EMNLP 2020 Pretrained Language Model Embryology: The Birth of ALBERT EMNLP 2020 Pre-training Entity Relation Encoder with Intra-span and Inter-span Information EMNLP 2020 Pre-training Multilingual Neural Machine Translation by Leveraging Alignment Information EMNLP 2020 Pre-training Tasks for Embedding-based Large-scale Retrieval ICLR 2020 Pre-Training Transformers as Energy-Based Cloze Models EMNLP 2020 Pretraining with Contrastive Sentence Objectives Improves Discourse Performance of Language Models ACL 2020 ProphetNet: Predicting Future N-gram for Sequence-to-SequencePre-training EMNLP 2020 Rare Words: A Major Problem for Contextualized Embeddings and How to Fix it by Attentive Mimicking AAAI 2020 Reading the Manual: Event Extraction as Definition Comprehension EMNLP 2020 Representations of Syntax [MASK] Useful: Effects of Constituency and Dependency Structure in Recursive LSTMs ACL 2020 Rethinking Computer-Aided Tuberculosis Diagnosis CVPR 2020 Rethinking the Hyperparameters for Fine-tuning ICLR 2020 Retiarii: A Deep Learning Exploratory-Training Framework OSDI 2020 Revisiting Pre-Trained Models for Chinese Natural Language Processing EMNLP 2020