Neural Network Optimization

On Lazy Training in Differentiable Programming NeurIPS 2019 On Model Stability as a Function of Random Seed CoNLL 2019 On the convergence of single-call stochastic extra-gradient methods NeurIPS 2019 On the Convergence of Stochastic Gradient Descent with Adaptive Stepsizes AISTATS 2019 On the Convergence of (Stochastic) Gradient Descent with Extrapolation for Non-Convex Minimization IJCAI 2019 On the Convergence Rate of Training Recurrent Neural Networks NeurIPS 2019 On the Dynamics of Gradient Descent for Autoencoders AISTATS 2019 On the Impact of the Activation function on Deep Neural Networks Training ICML 2019 On the Ineffectiveness of Variance Reduced Optimization for Deep Learning NeurIPS 2019 On the Realization of Compositionality in Neural Networks ACL 2019 On the Relation Between the Sharpest Directions of DNN Loss and the SGD Step Length ICLR 2019 Optimal Kronecker-Sum Approximation of Real Time Recurrent Learning ICML 2019 Optimization of Inf-Convolution Regularized Nonconvex Composite Problems AISTATS 2019 Optimizing Network Structure for 3D Human Pose Estimation ICCV 2019 Overcoming Multi-model Forgetting ICML 2019 Overparameterized Nonlinear Learning: Gradient Descent Takes the Shortest Path? ICML 2019 PA-GD: On the Convergence of Perturbed Alternating Gradient Descent to Second-Order Stationary Points for Structured Nonconvex Optimization ICML 2019 Painless Stochastic Gradient: Interpolation, Line-Search, and Convergence Rates NeurIPS 2019 Parallelizable Stack Long Short-Term Memory NAACL 2019 Popularity Prediction on Online Articles with Deep Fusion of Temporal Process and Content Features AAAI 2019 Positional Encoding to Control Output Sequence Length NAACL 2019 Practical Deep Learning with Bayesian Principles NeurIPS 2019 Preconditioner on Matrix Lie Group for SGD ICLR 2019 Predicting the Generalization Gap in Deep Networks with Margin Distributions ICLR 2019 Progressive Differentiable Architecture Search: Bridging the Depth Gap Between Search and Evaluation ICCV 2019