Pretraining

KD-VLP: Improving End-to-End Vision-and-Language Pretraining with Object Knowledge Distillation NAACL 2022 Knowledge-Augmented Language Models for Cause-Effect Relation Classification ACL 2022 Knowledge Distillation For CTC-based Speech Recognition Via Consistent Acoustic Representation Learning Interspeech 2022 Knowledge Distillation With the Reused Teacher Classifier CVPR 2022 Label Semantic Aware Pre-training for Few-shot Text Classification ACL 2022 LAION-5B: An open large-scale dataset for training next generation image-text models NeurIPS 2022 Language-biased image classification: evaluation based on semantic representations ICLR 2022 LastResort at SemEval-2022 Task 5: Towards Misogyny Identification using Visual Linguistic Model Ensembles And Task-Specific Pretraining SemEval 2022 Late Prompt Tuning: A Late Prompt Could Be Better Than Many Prompts EMNLP 2022 Learning to Deblur Using Light Field Generated and Real Defocus Images CVPR 2022 Learning to Retrieve Passages without Supervision NAACL 2022 LEMON: Language-Based Environment Manipulation via Execution-Guided Pre-training EMNLP 2022 Less Descriptive yet Discriminative: Quantifying the Properties of Multimodal Referring Utterances via CLIP ACL 2022 LFPT5: A Unified Framework for Lifelong Few-shot Language Learning Based on Prompt Tuning of T5 ICLR 2022 LightHuBERT: Lightweight and Configurable Speech Representation Learning with Once-for-All Hidden-Unit BERT Interspeech 2022 LILE: Look In-Depth before Looking Elsewhere – A Dual Attention Network using Transformers for Cross-Modal Information Retrieval in Histopathology Archives MIDL 2022 LTRC @ Causal News Corpus 2022: Extracting and Identifying Causal Elements using Adapters EMNLP 2022 MAE-AST: Masked Autoencoding Audio Spectrogram Transformer Interspeech 2022 MarkupLM: Pre-training of Text and Markup Language for Visually Rich Document Understanding ACL 2022 Masked Autoencoders Are Scalable Vision Learners CVPR 2022 MaxMatch-Dropout: Subword Regularization for WordPiece COLING 2022 MCP: Self-supervised Pre-training for Personalized Chatbots with Multi-level Contrastive Sampling EMNLP 2022 MERLOT Reserve: Neural Script Knowledge Through Vision and Language and Sound CVPR 2022 Meta-Auto-Decoder for Solving Parametric Partial Differential Equations NeurIPS 2022 MGDoc: Pre-training with Multi-granular Hierarchy for Document Image Understanding EMNLP 2022