Neural Network Optimization

Sparse linear networks with a fixed butterfly structure: theory and practice UAI 2021 Spectral Normalisation for Deep Reinforcement Learning: An Optimisation Perspective ICML 2021 Stochastic Whitening Batch Normalization CVPR 2021 Successive Halving Top-k Operator AAAI 2021 Teacher Guided Neural Architecture Search for Face Recognition AAAI 2021 Tensor Normal Training for Deep Learning Models NeurIPS 2021 Tensor Programs IIb: Architectural Universality Of Neural Tangent Kernel Training Dynamics ICML 2021 The Effects of Mild Over-parameterization on the Optimization Landscape of Shallow ReLU Neural Networks COLT 2021 The Heavy-Tail Phenomenon in SGD ICML 2021 The Implicit Bias of Minima Stability: A View from Function Space NeurIPS 2021 The Spectrum of Fisher Information of Deep Networks Achieving Dynamical Isometry AISTATS 2021 THOR, Trace-based Hardware-driven Layer-Oriented Natural Gradient Descent Computation AAAI 2021 Tightening Robustness Verification of Convolutional Neural Networks with Fine-Grained Linear Approximation AAAI 2021 Time Adaptive Recurrent Neural Network CVPR 2021 Towards Trustworthy Predictions from Deep Neural Networks with Fast Adversarial Calibration AAAI 2021 Train simultaneously, generalize better: Stability of gradient-based minimax learners ICML 2021 TRQ: Ternary Neural Networks With Residual Quantization AAAI 2021 Unchain the Search Space with Hierarchical Differentiable Architecture Search AAAI 2021 Vid-ODE: Continuous-Time Video Generation with Neural Ordinary Differential Equation AAAI 2021 When Are Solutions Connected in Deep Networks? NeurIPS 2021 When does gradient descent with logistic loss interpolate using deep networks with smoothed ReLU activations? COLT 2021 Zero Time Waste: Recycling Predictions in Early Exit Neural Networks NeurIPS 2021 Absum: Simple Regularization Method for Reducing Structural Sensitivity of Convolutional Neural Networks AAAI 2020 Adaptive Convolutional ReLUs AAAI 2020 A Mean Field Analysis Of Deep ResNet And Beyond: Towards Provably Optimization Via Overparameterization From Depth ICML 2020