Neural Network Optimization

Fast Abstractive Summarization with Reinforce-Selected Sentence Rewriting ACL 2018 Fast and Scalable Bayesian Deep Learning by Weight-Perturbation in Adam ICML 2018 Fast Approximate Natural Gradient Descent in a Kronecker Factored Eigenbasis NeurIPS 2018 Faster Derivative-Free Stochastic Algorithm for Shared Memory Machines ICML 2018 Fast Gradient-Based Methods with Exponential Rate: A Hybrid Control Framework ICML 2018 FastGRNN: A Fast, Accurate, Stable and Tiny Kilobyte Sized Gated Recurrent Neural Network NeurIPS 2018 First-order Stochastic Algorithms for Escaping From Saddle Points in Almost Linear Time NeurIPS 2018 FLAG n’ FLARE: Fast Linearly-Coupled Adaptive Gradient Methods AISTATS 2018 From Credit Assignment to Entropy Regularization: Two New Algorithms for Neural Sequence Prediction ACL 2018 Generating Reasonable and Diversified Story Ending Using Sequence to Sequence Model with Adversarial Training COLING 2018 Global Convergence of Langevin Dynamics Based Algorithms for Nonconvex Optimization NeurIPS 2018 Gradient Descent for Spiking Neural Networks NeurIPS 2018 Gradient Descent Learns Linear Dynamical Systems JMLR 2018 Gradient Descent Learns One-hidden-layer CNN: Don’t be Afraid of Spurious Local Minima ICML 2018 Gradient descent with identity initialization efficiently learns positive definite linear transformations by deep residual networks ICML 2018 Gradient Primal-Dual Algorithm Converges to Second-Order Stationary Solution for Nonconvex Distributed Optimization Over Networks ICML 2018 GradNorm: Gradient Normalization for Adaptive Loss Balancing in Deep Multitask Networks ICML 2018 Guaranteed Sufficient Decrease for Stochastic Variance Reduced Gradient Optimization AISTATS 2018 Hessian-based Analysis of Large Batch Training and Robustness to Adversaries NeurIPS 2018 How Does Batch Normalization Help Optimization? NeurIPS 2018 How SGD Selects the Global Minima in Over-parameterized Learning: A Dynamical Stability Perspective NeurIPS 2018 How to Start Training: The Effect of Initialization and Architecture NeurIPS 2018 Hybrid Macro/Micro Level Backpropagation for Training Deep Spiking Neural Networks NeurIPS 2018 Image Correction via Deep Reciprocating HDR Transformation CVPR 2018 Implicit Bias of Gradient Descent on Linear Convolutional Networks NeurIPS 2018