gradient descent

A Dynamical Model of Neural Scaling Laws ICML 2024 A generalized neural tangent kernel for surrogate gradient learning NeurIPS 2024 A Hierarchical Bayesian Model for Few-Shot Meta Learning ICLR 2024 Aligning Individual and Collective Objectives in Multi-Agent Cooperation NeurIPS 2024 An Accelerated Algorithm for Stochastic Bilevel Optimization under Unbounded Smoothness NeurIPS 2024 Analyzing & Reducing the Need for Learning Rate Warmup in GPT Training NeurIPS 2024 An Efficient Stochastic Algorithm for Decentralized Nonconvex-Strongly-Concave Minimax Optimization AISTATS 2024 An operator preconditioning perspective on training in physics-informed machine learning ICLR 2024 A PID Controller Approach for Adaptive Probability-dependent Gradient Decay in Model Calibration NeurIPS 2024 Approximated Orthogonal Projection Unit: Stabilizing Regression Network Training Using Natural Gradient NeurIPS 2024 A Theory of Non-Linear Feature Learning with One Gradient Step in Two-Layer Neural Networks ICML 2024 Batch Normalization Is Blind to the First and Second Derivatives of the Loss AAAI 2024 Bayes-optimal learning of an extensive-width neural network from quadratically many samples NeurIPS 2024 Beating Price of Anarchy and Gradient Descent without Regret in Potential Games ICLR 2024 Benign Overfitting and Grokking in ReLU Networks for XOR Cluster Data ICLR 2024 Benign Overfitting in Two-Layer ReLU Convolutional Neural Networks for XOR Data ICML 2024 Byzantine-Robust Decentralized Learning via Remove-then-Clip Aggregation AAAI 2024 Can Gaussian Sketching Converge Faster on a Preconditioned Landscape? ICML 2024 Can Looped Transformers Learn to Implement Multi-step Gradient Descent for In-context Learning? ICML 2024 Changing the Training Data Distribution to Reduce Simplicity Bias Improves In-distribution Generalization NeurIPS 2024 CO2: Efficient Distributed Training with Full Communication-Computation Overlap ICLR 2024 Communication-Efficient Gradient Descent-Accent Methods for Distributed Variational Inequalities: Unified Analysis and Local Updates ICLR 2024 Compression of Structured Data with Autoencoders: Provable Benefit of Nonlinearities and Depth ICML 2024 Conflict-Alleviated Gradient Descent for Adaptive Object Detection IJCAI 2024 Conformal Prediction with Learned Features ICML 2024