gradient descent

Towards Unified Representations of Knowledge Graph and Expert Rules for Machine Learning and Reasoning AACL 2022 Training High-Performance Low-Latency Spiking Neural Networks by Differentiation on Spike Representation CVPR 2022 Training Two-Layer ReLU Networks with Gradient Descent is Inconsistent JMLR 2022 Transition to Linearity of Wide Neural Networks is an Emerging Property of Assembling Weak Models ICLR 2022 Understanding approximate and unrolled dictionary learning for pattern recovery ICLR 2022 Understanding Gradient Descent on the Edge of Stability in Deep Learning ICML 2022 Understanding the Generalization Benefit of Normalization Layers: Sharpness Reduction NeurIPS 2022 Vanishing Curvature in Randomly Initialized Deep ReLU Networks AISTATS 2022 Variational autoencoders in the presence of low-dimensional data: landscape and implicit bias ICLR 2022 What Can Transformers Learn In-Context? A Case Study of Simple Function Classes NeurIPS 2022 When Hardness of Approximation Meets Hardness of Learning JMLR 2022 Zeroth-Order Hard-Thresholding: Gradient Error vs. Expansivity NeurIPS 2022 α-IoU: A Family of Power Intersection over Union Losses for Bounding Box Regression NeurIPS 2021 A case where a spindly two-layer linear network decisively outperforms any neural network with a fully connected input layer ALT 2021 Accelerated Algorithms for Smooth Convex-Concave Minimax Problems with O(1/k^2) Rate on Squared Gradient Norm ICML 2021 Acceleration via Fractal Learning Rate Schedules ICML 2021 A Comprehensively Tight Analysis of Gradient Descent for PCA NeurIPS 2021 A Convergence Analysis of Gradient Descent on Graph Neural Networks NeurIPS 2021 AdamP: Slowing Down the Slowdown for Momentum Optimizers on Scale-invariant Weights ICLR 2021 Adversarial Regularization as Stackelberg Game: An Unrolled Optimization Approach EMNLP 2021 A Dynamical View on Optimization Algorithms of Overparameterized Neural Networks AISTATS 2021 Agnostic Learning of Halfspaces with Gradient Descent via Soft Margins ICML 2021 A Gradient Method for Multilevel Optimization NeurIPS 2021 A Hybrid Variance-Reduced Method for Decentralized Stochastic Non-Convex Optimization ICML 2021 A Local Convergence Theory for Mildly Over-Parameterized Two-Layer Neural Network COLT 2021