Pretraining

ProphetNet-X: Large-Scale Pre-training Models for English, Chinese, Multi-lingual, Dialog, and Code Generation ACL 2021 Read Like Humans: Autonomous, Bidirectional and Iterative Language Modeling for Scene Text Recognition CVPR 2021 Recent advances in neural metaphor processing: A linguistic, cognitive and social perspective NAACL 2021 [RETRACTED] Breaking the Corpus Bottleneck for Context-Aware Neural Machine Translation with Cross-Task Pre-training ACL 2021 Revisiting Pivot-Based Paraphrase Generation: Language Is Not the Only Optional Pivot EMNLP 2021 RoBLEURT Submission for WMT2021 Metrics Task EMNLP 2021 Scalable Transfer Learning with Expert Models ICLR 2021 Scaling Up Visual and Vision-Language Representation Learning With Noisy Text Supervision ICML 2021 Scientific Discourse Tagging for Evidence Extraction EACL 2021 Seasonal Contrast: Unsupervised Pre-Training From Uncurated Remote Sensing Data ICCV 2021 SEED: Self-supervised Distillation For Visual Representation ICLR 2021 Segatron: Segment-Aware Transformer for Language Modeling and Understanding AAAI 2021 Self-Supervised 3D Skeleton Action Representation Learning With Motion Consistency and Continuity ICCV 2021 SemFace: Pre-training Encoder and Decoder with a Semantic Interface for Neural Machine Translation ACL 2021 SemiHand: Semi-Supervised Hand Pose Estimation With Consistency ICCV 2021 SeqMatchNet: Contrastive Learning with Sequence Matching for Place Recognition & Relocalization CoRL 2021 SkoltechNLP at SemEval-2021 Task 5: Leveraging Sentence-level Pre-training for Toxic Span Detection SemEval 2021 Small Data? No Problem! Exploring the Viability of Pretrained Multilingual Language Models for Low-resourced Languages EMNLP 2021 SMedBERT: A Knowledge-Enhanced Pre-trained Language Model with Structured Semantics for Medical Text Mining ACL 2021 Space-Time-Separable Graph Convolutional Network for Pose Forecasting ICCV 2021 Span Fine-tuning for Pre-trained Language Models EMNLP 2021 Speak or Chat with Me: End-to-End Spoken Language Understanding System with Flexible Inputs Interspeech 2021 Speech Denoising with Auditory Models Interspeech 2021 StEP: Style-Based Encoder Pre-Training for Multi-Modal Image Synthesis CVPR 2021 StereOBJ-1M: Large-Scale Stereo Image Dataset for 6D Object Pose Estimation ICCV 2021