Neural Network Optimization

L4: Practical loss-based stepsize adaptation for deep learning NeurIPS 2018 Learning Compact Neural Networks with Regularization ICML 2018 Learning Overparameterized Neural Networks via Stochastic Gradient Descent on Structured Data NeurIPS 2018 Learning Time/Memory-Efficient Deep Architectures With Budgeted Super Networks CVPR 2018 MorphNet: Fast & Simple Resource-Constrained Structure Learning of Deep Networks CVPR 2018 On Acceleration with Noise-Corrupted Gradients ICML 2018 Online Learning Rate Adaptation with Hypergradient Descent ICLR 2018 On Neuronal Capacity NeurIPS 2018 On the Convergence Properties of a K-step Averaging Stochastic Gradient Descent Algorithm for Nonconvex Optimization IJCAI 2018 Optimization Algorithm Inspired Deep Neural Network Structure Design ACML 2018 Searching for Efficient Multi-Scale Architectures for Dense Image Prediction NeurIPS 2018 Step Size Matters in Deep Learning NeurIPS 2018 The Physical Systems Behind Optimization Algorithms NeurIPS 2018 Training Deep Neural Networks with 8-bit Floating Point Numbers NeurIPS 2018 Training DNNs with Hybrid Block Floating Point NeurIPS 2018 Accelerated Doubly Stochastic Gradient Algorithm for Large-scale Empirical Risk Minimization IJCAI 2017 Active Bias: Training More Accurate Neural Networks by Emphasizing High Variance Samples NeurIPS 2017 All You Need Is Beyond a Good Init: Exploring Better Solution for Training Extremely Deep Convolutional Neural Networks With Orthonormality and Modulation CVPR 2017 Batch Renormalization: Towards Reducing Minibatch Dependence in Batch-Normalized Models NeurIPS 2017 Convergent Block Coordinate Descent for Training Tikhonov Regularized Deep Neural Networks NeurIPS 2017 Follow the Moving Leader in Deep Learning ICML 2017 Gated Recurrent Convolution Neural Network for OCR NeurIPS 2017 Improving Training of Deep Neural Networks via Singular Value Bounding CVPR 2017 Input-to-Output Gate to Improve RNN Language Models IJCNLP 2017 Learning Adaptive Receptive Fields for Deep Image Parsing Network CVPR 2017