conftrace_

learning rate scheduler

4 papers

Explore in graph

Co-occurring keywords

supernet training (26) multi-task optimization (13) gradient difference (3) large language model (13587) normalized gradient descent (3) dynamic training strategy (1) momentum separation (1) dynamic training (6) training dynamics (118) model convergence (39)

Papers

Subnet-Aware Dynamic Supernet Training for Neural Architecture Search CVPR 2025
Unlearning as multi-task optimization: A normalized gradient difference approach with an adaptive learning rate NAACL 2025
Hop, skip, jump to Convergence: Dynamics of Learning Rate Transitions for Improved Training of Large Language Models EMNLP 2024
Mind the (optimality) Gap: A Gap-Aware Learning Rate Scheduler for Adversarial Nets AISTATS 2023