Pretraining

XDoc: Unified Pre-training for Cross-Format Document Understanding EMNLP 2022 XLM-K: Improving Cross-Lingual Language Model Pre-training with Multilingual Knowledge AAAI 2022 XLS-R: Self-supervised Cross-lingual Speech Representation Learning at Scale Interspeech 2022 XMP-Font: Self-Supervised Cross-Modality Pre-Training for Few-Shot Font Generation CVPR 2022 Yes, No or IDK: The Challenge of Unanswerable Yes/No Questions NAACL 2022 YMAI at SemEval-2022 Task 5: Detecting Misogyny in Memes using VisualBERT and MMBT MultiModal Pre-trained Models SemEval 2022 You Only Cut Once: Boosting Data Augmentation with a Single Cut ICML 2022 ZeroPrompt: Scaling Prompt-Based Pretraining to 1,000 Tasks Improves Zero-Shot Generalization EMNLP 2022 Zero-Shot Cross-Lingual Machine Reading Comprehension via Inter-sentence Dependency Graph AAAI 2022 Zero-Shot Out-of-Distribution Detection Based on the Pre-trained Model CLIP AAAI 2022 Achieving Forgetting Prevention and Knowledge Transfer in Continual Learning NeurIPS 2021 ActionBert: Leveraging User Actions for Semantic Understanding of User Interfaces AAAI 2021 Adaptive Text to Speech for Spontaneous Style Interspeech 2021 AdaptSum: Towards Low-Resource Domain Adaptation for Abstractive Summarization NAACL 2021 A Hands-On Comparison of DNNs for Dialog Separation Using Transfer Learning from Music Source Separation Interspeech 2021 A Little Pretraining Goes a Long Way: A Case Study on Dependency Parsing Task for Low-resource Morphologically Rich Languages EACL 2021 A Million Tweets Are Worth a Few Points: Tuning Transformers for Customer Service Tasks NAACL 2021 AMuSE-WSD: An All-in-one Multilingual System for Easy Word Sense Disambiguation EMNLP 2021 An Efficient Streaming Non-Recurrent On-Device End-to-End Model with Improvements to Rare-Word Modeling Interspeech 2021 An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale ICLR 2021 An Improved Wav2Vec 2.0 Pre-Training Approach Using Enhanced Local Dependency Modeling for Speech Recognition Interspeech 2021 Annealing Knowledge Distillation EACL 2021 Anytime Inference with Distilled Hierarchical Neural Ensembles AAAI 2021 APGN: Adversarial and Parameter Generation Networks for Multi-Source Cross-Domain Dependency Parsing EMNLP 2021 A Pre-training Strategy for Zero-Resource Response Selection in Knowledge-Grounded Conversations ACL 2021