← Techniques
Deep Learning › Techniques ›

Pretraining

2471 directly classified papers

Papers per year

Papers

How does the task complexity of masked pretraining objectives affect downstream performance? ACL 2023
Revisit Few-shot Intent Classification with PLMs: Direct Fine-tuning vs. Continual Pre-training ACL 2023
A Study on Knowledge Distillation from Weak Teacher for Scaling Up Pre-trained Language Models ACL 2023
On the Difference of BERT-style and CLIP-style Text Encoders ACL 2023
Multi-Source (Pre-)Training for Cross-Domain Measurement, Unit and Context Extraction ACL 2023
Navigating Data Scarcity: Pretraining for Medical Utterance Classification ACL 2023
I2R’s End-to-End Speech Translation System for IWSLT 2023 Offline Shared Task ACL 2023
SPC: Soft Prompt Construction for Cross Domain Generalization ACL 2023
Leveraging Pretrained ASR Encoders for Effective and Efficient End-to-End Speech Intent Classification and Slot Filling INTERSPEECH 2023
A Teacher-Student Approach for Extracting Informative Speaker Embeddings From Speech Mixtures INTERSPEECH 2023
Expresso: A Benchmark and Analysis of Discrete Expressive Speech Resynthesis INTERSPEECH 2023
MMSpeech: Multi-modal Multi-task Encoder-Decoder Pre-training for speech recognition INTERSPEECH 2023
wav2vec 2.0 ASR for Cantonese-Speaking Older Adults in a Clinical Setting INTERSPEECH 2023
N-Shot Benchmarking of Whisper on Diverse Arabic Speech Recognition INTERSPEECH 2023
The MALACH Corpus: Results with End-to-End Architectures and Pretraining INTERSPEECH 2023
Diffusion-based Visual Representation Learning for Medical Question Answering ACML 2023
One-Line-of-Code Data Mollification Improves Optimization of Likelihood-based Generative Models NIPS 2023
Prompt Pre-Training with Twenty-Thousand Classes for Open-Vocabulary Visual Recognition NIPS 2023
Make Pre-trained Model Reversible: From Parameter to Memory Efficient Fine-Tuning NIPS 2023
MosaicBERT: A Bidirectional Encoder Optimized for Fast Pretraining NIPS 2023
Meet in the Middle: A New Pre-training Paradigm NIPS 2023
Does progress on ImageNet transfer to real-world datasets? NIPS 2023
RevColV2: Exploring Disentangled Representations in Masked Image Modeling NIPS 2023
Battle of the Backbones: A Large-Scale Comparison of Pretrained Models across Computer Vision Tasks NIPS 2023
The Emergence of Essential Sparsity in Large Pre-trained Models: The Weights that Matter NIPS 2023