conftrace_

hyperparameter transfer

11 papers

Explore in graph

Co-occurring keywords · papers mentioning both

neural network optimization (4) residual network (2) feature learning (2) transformer operations (1) network dynamics (1) sharpness-aware minimization (1) gradient descent (1) second-order optimization (1) multi-armed bandit (1) learning rate (1)

Papers

$\mathrmμ$nit Scaling: Simple and Scalable FP8 LLM Training ICML 2025 Deep Linear Network Training Dynamics from Random Initialization: Data, Width, Depth, and Hyperparameter Transfer ICML 2025 Function-Space Learning Rates ICML 2025 Maximal Update Parametrization and Zero-Shot Hyperparameter Transfer for Fourier Neural Operators ICML 2025 Offline-to-Online Hyperparameter Transfer for Stochastic Bandits AAAI 2025 Scaling Optimal LR Across Token Horizons ICLR 2025 $\boldsymbol{\mu}\mathbf{P^2}$: Effective Sharpness Aware Minimization Requires Layerwise Perturbation Scaling NeurIPS 2024 Depthwise Hyperparameter Transfer in Residual Networks: Dynamics and Scaling Limit ICLR 2024 On the Parameterization of Second-Order Optimization Effective towards the Infinite Width ICLR 2024 Sparse maximal update parameterization: A holistic approach to sparse training dynamics NeurIPS 2024 Tensor Programs VI: Feature Learning in Infinite Depth Neural Networks ICLR 2024