Neural Network Optimization

Stochastic Gradient and Langevin Processes ICML 2020 Stochastic Gradient Descent in Correlated Settings: A Study on Gaussian Processes NeurIPS 2020 Stochasticity of Deterministic Gradient Descent: Large Learning Rate for Multiscale Objective Function NeurIPS 2020 Stochastic Loss Function AAAI 2020 Stochastic Optimization with Heavy-Tailed Noise via Accelerated Gradient Clipping NeurIPS 2020 Stochastic Recursive Gradient Descent Ascent for Stochastic Nonconvex-Strongly-Concave Minimax Problems NeurIPS 2020 Streaming Batch Gradient Tracking for Neural Network Training (Student Abstract) AAAI 2020 Structured Sparsification of Gated Recurrent Neural Networks AAAI 2020 Successfully Applying the Stabilized Lottery Ticket Hypothesis to the Transformer Architecture ACL 2020 Super-efficiency of automatic differentiation for functions defined as a minimum ICML 2020 Synchronous Double-channel Recurrent Network for Aspect-Opinion Pair Extraction ACL 2020 Target Propagation in Recurrent Neural Networks JMLR 2020 Task-Level Curriculum Learning for Non-Autoregressive Neural Machine Translation IJCAI 2020 Taylor Expansion Policy Optimization ICML 2020 TeaForN: Teacher-Forcing with N-grams EMNLP 2020 Template-Based Automatic Search of Compact Semantic Segmentation Architectures WACV 2020 The Break-Even Point on Optimization Trajectories of Deep Neural Networks ICLR 2020 The Devil Is in the Details: Delving Into Unbiased Data Processing for Human Pose Estimation CVPR 2020 The Early Phase of Neural Network Training ICLR 2020 The EOS Decision and Length Extrapolation EMNLP 2020 The Generalization-Stability Tradeoff In Neural Network Pruning NeurIPS 2020 The HSIC Bottleneck: Deep Learning without Back-Propagation AAAI 2020 The Implicit Bias of Depth: How Incremental Learning Drives Generalization ICLR 2020 The Implicit Regularization of Stochastic Gradient Flow for Least Squares ICML 2020 The intriguing role of module criticality in the generalization of deep networks ICLR 2020