Skip to content
conftrace_

learning rate tuning

4 papers

Explore in graph

Co-occurring keywords · papers mentioning both

optimizer comparison (1) fine-tuning transformer (1) loss hessian (1) parameter distance (1) gradient breakdown (1) deep relative trust (1) compositional function (1) multiplicative weight update (1) adaptive optimizer (1) learning rate (1)

Papers

Should I try multiple optimizers when fine-tuning a pre-trained Transformer for NLP tasks? Should I tune their hyperparameters? EACL 2024 Stepping on the Edge: Curvature Aware Learning Rate Tuners NeurIPS 2024 Learning compositional functions via multiplicative weight updates NeurIPS 2020 On the distance between two neural networks and the stability of learning NeurIPS 2020