← Techniques
Deep Learning › Techniques ›

Pretraining

2471 directly classified papers

Papers per year

Papers

ArchBERT: Bi-Modal Understanding of Neural Architectures and Natural Languages CONLL 2023
DropPos: Pre-Training Vision Transformers by Reconstructing Dropped Positions NIPS 2023
GAFNet: A Global Fourier Self Attention Based Novel Network for Multi-Modal Downstream Tasks WACV 2023
Good Helper Is around You: Attention-Driven Masked Image Modeling AAAI 2023
PeCo: Perceptual Codebook for BERT Pre-training of Vision Transformers AAAI 2023
Investigating Pre-trained Audio Encoders in the Low-Resource Condition INTERSPEECH 2023
MGMAE: Motion Guided Masking for Video Masked Autoencoding ICCV 2023
XPhoneBERT: A Pre-trained Multilingual Model for Phoneme Representations for Text-to-Speech INTERSPEECH 2023
Accelerating Trajectory Generation for Quadrotors Using Transformers L4DC 2023
Understanding Spoken Language Development of Children with ASD Using Pre-trained Speech Embeddings INTERSPEECH 2023
Misalign, Contrast then Distill: Rethinking Misalignments in Language-Image Pre-training ICCV 2023
Pre-training Vision Transformers with Very Limited Synthesized Images ICCV 2023
AD-PT: Autonomous Driving Pre-Training with Large-scale Point Cloud Dataset NIPS 2023
MultiFusion: Fusing Pre-Trained Models for Multi-Lingual, Multi-Modal Image Generation NIPS 2023
StyleDrop: Text-to-Image Synthesis of Any Style NIPS 2023
Quality Diversity for Visual Pre-Training ICCV 2023
Adaptation of Text-Conditioned Diffusion Models for Audio-to-Image Generation INTERSPEECH 2023
Reducing Barriers to Self-Supervised Learning: HuBERT Pre-training with Academic Compute INTERSPEECH 2023
More Than Just Attention: Improving Cross-Modal Attentions With Contrastive Constraints for Image-Text Matching WACV 2023
Do Pre-Trained Models Benefit Equally in Continual Learning? WACV 2023
Model-Generated Pretraining Signals Improves Zero-Shot Generalization of Text-to-Text Transformers ACL 2023
An Empirical Investigation of the Role of Pre-training in Lifelong Learning JMLR 2023
Blessing of Class Diversity in Pre-training AISTATS 2023
DiffS2UT: A Semantic Preserving Diffusion Model for Textless Direct Speech-to-Speech Translation EMNLP 2023
CodeFusion: A Pre-trained Diffusion Model for Code Generation EMNLP 2023