Neural Network Optimization

Adam-family Methods for Nonsmooth Optimization with Convergence Guarantees JMLR 2024 Adam on Local Time: Addressing Nonstationarity in RL with Relative Adam Timesteps NeurIPS 2024 Adam with model exponential moving average is effective for nonconvex optimization NeurIPS 2024 Adaptive Quasi-Newton and Anderson Acceleration Framework with Explicit Global (Accelerated) Convergence Rates AISTATS 2024 Adaptive Regularization of Representation Rank as an Implicit Constraint of Bellman Equation ICLR 2024 Adaptive Sharpness-Aware Pruning for Robust Sparse Networks ICLR 2024 AdaRevD: Adaptive Patch Exiting Reversible Decoder Pushes the Limit of Image Deblurring CVPR 2024 Addressing Loss of Plasticity and Catastrophic Forgetting in Continual Learning ICLR 2024 ADOPT: Modified Adam Can Converge with Any β_2 with the Optimal Rate NeurIPS 2024 ADR-X: ANN-Assisted Wireless Link Rate Adaptation for Compute-Constrained Embedded Gaming Devices NSDI 2024 Advancing Training Efficiency of Deep Spiking Neural Networks through Rate-based Backpropagation NeurIPS 2024 Adversarial Initialization with Universal Adversarial Perturbation: A New Approach to Fast Adversarial Training AAAI 2024 A General and Efficient Training for Transformer via Token Expansion CVPR 2024 A generalized neural tangent kernel for surrogate gradient learning NeurIPS 2024 A Layer-Wise Natural Gradient Optimizer for Training Deep Neural Networks NeurIPS 2024 A Learning Rate Path Switching Training Paradigm for Version Updates of Large Language Models EMNLP 2024 Algorithmic progress in language models NeurIPS 2024 Almost Sure Convergence Rates Analysis and Saddle Avoidance of Stochastic Gradient Methods JMLR 2024 Alternating minimization for generalized rank one matrix sensing: Sharp predictions from a random initialization ALT 2024 AmoebaLLM: Constructing Any-Shape Large Language Models for Efficient and Instant Deployment NeurIPS 2024 Analysing The Impact of Sequence Composition on Language Model Pre-Training ACL 2024 Analyzing and Improving the Training Dynamics of Diffusion Models CVPR 2024 Analyzing Feed-Forward Blocks in Transformers through the Lens of Attention Maps ICLR 2024 Analyzing Few-Shot Neural Architecture Search in a Metric-Driven Framework AutoML 2024 Analyzing Generalization in Policy Networks: A Case Study with the Double-Integrator System AAAI 2024