← Keywords

model pretraining

35 papers

Explore in graph

Co-occurring keywords

transfer learning (5442) self-supervised learning (3751) representation learning (6174) language model (4573) language model pretraining (105) knowledge distillation (3680) masked image modeling (83) object detection (2759) zero-shot learning (3637) model compression (3283)

Papers

Overlapping Context with Variable-Length Stride Increases Diversity when Training Large Language Model for Code ACL 2025
Learning Dynamics of Meta-Learning in Small Model Pretraining AACL 2025
An Empirical Comparison of Vocabulary Expansion and Initialization Approaches For Language Models CONLL 2024
SFOD: Spiking Fusion Object Detector CVPR 2024
On the Inductive Bias of Stacking Towards Improving Reasoning NIPS 2024
Scaling Parameter-Constrained Language Models with Quality Data EMNLP 2024
Pre-training Differentially Private Models with Limited Public Data NIPS 2024
DeepSpeed Data Efficiency: Improving Deep Learning Model Quality and Training Efficiency via Efficient Data Sampling and Routing AAAI 2024
ParameterNet: Parameters Are All You Need for Large-scale Visual Pretraining of Mobile Networks CVPR 2024
Characterization of Large Language Model Development in the Datacenter NSDI 2024
EHRSHOT: An EHR Benchmark for Few-Shot Evaluation of Foundation Models NIPS 2023
Gold-YOLO: Efficient Object Detector via Gather-and-Distribute Mechanism NIPS 2023
Revealing the Dark Secrets of Masked Image Modeling CVPR 2023
Pre-training Multi-party Dialogue Models with Latent Discourse Inference ACL 2023
The Effectiveness of MAE Pre-Pretraining for Billion-Scale Pretraining ICCV 2023
Unleashing Vanilla Vision Transformer with Masked Image Modeling for Object Detection ICCV 2023
Stare at What You See: Masked Image Modeling Without Reconstruction CVPR 2023
Towards Unifying Multi-Lingual and Cross-Lingual Summarization ACL 2023
Recipes for Sequential Pre-training of Multilingual Encoder and Seq2Seq Models ACL 2023
Investigating Self-supervised Pretraining Frameworks for Pathological Speech Recognition INTERSPEECH 2022
Exploring Capabilities of Monolingual Audio Transformers using Large Datasets in Automatic Speech Recognition of Czech INTERSPEECH 2022
Pre-Train, Self-Train, Distill: A Simple Recipe for Supersizing 3D Reconstruction CVPR 2022
Masked Autoencoding for Scalable and Generalizable Decision Making NIPS 2022
Lifelong Pretraining: Continually Adapting Language Models to Emerging Corpora ACL 2022
Towards Unsupervised Domain Generalization CVPR 2022