Neural Network Optimization

Adaptive Depth Networks with Skippable Sub-Paths NeurIPS 2024 Adaptive Rational Activations to Boost Deep Reinforcement Learning ICLR 2024 ADOPT: Modified Adam Can Converge with Any β_2 with the Optimal Rate NeurIPS 2024 ADR-X: ANN-Assisted Wireless Link Rate Adaptation for Compute-Constrained Embedded Gaming Devices NSDI 2024 A Geometry Loss Combination for 3D Human Pose Estimation WACV 2024 A Layer-Wise Natural Gradient Optimizer for Training Deep Neural Networks NeurIPS 2024 Almost Sure Convergence Rates Analysis and Saddle Avoidance of Stochastic Gradient Methods JMLR 2024 An Accelerated Algorithm for Stochastic Bilevel Optimization under Unbounded Smoothness NeurIPS 2024 Analysing The Impact of Sequence Composition on Language Model Pre-Training ACL 2024 Analyzing and Improving the Training Dynamics of Diffusion Models CVPR 2024 Analyzing & Reducing the Need for Learning Rate Warmup in GPT Training NeurIPS 2024 An Efficient High-dimensional Gradient Estimator for Stochastic Differential Equations NeurIPS 2024 Approximated Orthogonal Projection Unit: Stabilizing Regression Network Training Using Natural Gradient NeurIPS 2024 A Single-Step, Sharpness-Aware Minimization is All You Need to Achieve Efficient and Accurate Sparse Training NeurIPS 2024 Attention-Driven Training-Free Efficiency Enhancement of Diffusion Models CVPR 2024 AutoDrop: Training Deep Learning Models with Automatic Learning Rate Drop UAI 2024 Batch Normalization Alleviates the Spectral Bias in Coordinate Networks CVPR 2024 Bayes-optimal learning of an extensive-width neural network from quadratically many samples NeurIPS 2024 Benign overfitting in leaky ReLU networks with moderate input dimension NeurIPS 2024 Bias in Motion: Theoretical Insights into the Dynamics of Bias in SGD Training NeurIPS 2024 BMRS: Bayesian Model Reduction for Structured Pruning NeurIPS 2024 CE-NAS: An End-to-End Carbon-Efficient Neural Architecture Search Framework NeurIPS 2024 Changing the Training Data Distribution to Reduce Simplicity Bias Improves In-distribution Generalization NeurIPS 2024 Consistency Diffusion Bridge Models NeurIPS 2024 ConsistentEE: A Consistent and Hardness-Guided Early Exiting Method for Accelerating Language Models Inference AAAI 2024