Pretraining

Compressed Video Contrastive Learning NeurIPS 2021 Conceptual 12M: Pushing Web-Scale Image-Text Pre-Training To Recognize Long-Tail Visual Concepts CVPR 2021 Condenser: a Pre-training Architecture for Dense Retrieval EMNLP 2021 Contextualized Streaming End-to-End Speech Recognition with Trie-Based Deep Biasing and Shallow Fusion Interspeech 2021 Contrastive Code Representation Learning EMNLP 2021 ConViT: Improving Vision Transformers with Soft Convolutional Inductive Biases ICML 2021 Convolutions and Self-Attention: Re-interpreting Relative Positions in Pre-trained Language Models ACL 2021 Counts@IITK at SemEval-2021 Task 8: SciBERT Based Entity And Semantic Relation Extraction For Scientific Data SemEval 2021 Cross-lingual Visual Pre-training for Multimodal Machine Translation EACL 2021 CRSLab: An Open-Source Toolkit for Building Conversational Recommender System ACL 2021 CS-BERT: a pretrained model for customer service dialogues EMNLP 2021 CS-UM6P at SemEval-2021 Task 1: A Deep Learning Model-based Pre-trained Transformer Encoder for Lexical Complexity SemEval 2021 DAP: Detection-Aware Pre-Training With Weak Supervision CVPR 2021 Data Augmentation for Voice-Assistant NLU using BERT-based Interchangeable Rephrase EACL 2021 Data-Efficient Language Shaped Few-shot Image Classification EMNLP 2021 Dense Keypoints via Multiview Supervision NeurIPS 2021 DepthInSpace: Exploitation and Fusion of Multiple Video Frames for Structured-Light Depth Estimation ICCV 2021 Differentially Private Learning Needs Better Features (or Much More Data) ICLR 2021 DirectQE: Direct Pretraining for Machine Translation Quality Estimation AAAI 2021 DiscoBox: Weakly Supervised Instance Segmentation and Semantic Correspondence From Box Supervision ICCV 2021 Discovering Representation Sprachbund For Multilingual Pre-Training EMNLP 2021 Distributed Deep Learning In Open Collaborations NeurIPS 2021 Diverse Pretrained Context Encodings Improve Document Translation ACL 2021 Does Pretraining for Summarization Require Knowledge Transfer? EMNLP 2021 Domain-Adaptive Pretraining Methods for Dialogue Understanding ACL 2021