conftrace_

model initialization

32 papers

Explore in graph

Co-occurring keywords

few-shot learning (3398) transfer learning (5449) model compression (3302) weight initialization (50) neural network optimization (1293) knowledge distillation (3725) variable-sized model (4) neural network (6616) language model (4599) knowledge transfer (648)

Papers

Learngene: Inheritable ‘Genes’ in Intelligent Agents (Abstract Reprint) AAAI 2026
Differentially Private Learning Needs Better Model Initialization and Self-Distillation NAACL 2025
Optimizing Hidden Markov Language Models: An Empirical Study of Reparameterization and Initialization Techniques NAACL 2025
WAVE: Weight Templates for Adaptive Initialization of Variable-sized Models CVPR 2025
Slamming: Training a Speech Language Model on One GPU in a Day ACL 2025
Exploring Learngene via Stage-wise Weight Sharing for Initializing Variable-sized Models IJCAI 2024
XB-MAML: Learning Expandable Basis Parameters for Effective Meta-Learning with Wide Task Coverage AISTATS 2024
Cluster-Learngene: Inheriting Adaptive Clusters for Vision Transformers NIPS 2024
BAM! Just Like That: Simple and Efficient Parameter Upcycling for Mixture of Experts NIPS 2024
Initializing Variable-sized Vision Transformers from Learngene with Learnable Transformation NIPS 2024
Transformer as Linear Expansion of Learngene AAAI 2024
GEMS: Generating Efficient Meta-Subnets WACV 2023
Can We Scale Transformers to Predict Parameters of Diverse ImageNet Models? ICML 2023
Mimetic Initialization of Self-Attention Layers ICML 2023
A Bayesian Approach To Analysing Training Data Attribution In Deep Learning NIPS 2023
Magneto: A Foundation Transformer ICML 2023
A Natural Bias for Language Generation Models ACL 2023
Learngene: From Open-World to Your Learning Task AAAI 2022
Improving Generalization in Meta-learning via Task Augmentation ICML 2021
Optimizing Deeper Transformers on Small Datasets IJCNLP 2021
Optimizing Deeper Transformers on Small Datasets ACL 2021
Few-Round Learning for Federated Learning NIPS 2021
A Representation Learning Perspective on the Importance of Train-Validation Splitting in Meta-Learning ICML 2021
The IICT-Yverdon System for the WMT 2021 Unsupervised MT and Very Low Resource Supervised MT Task EMNLP 2021
Drawing Robust Scratch Tickets: Subnetworks with Inborn Robustness Are Found within Randomly Initialized Networks NIPS 2021