weight decay

Large Norms of CNN Layers Do Not Hurt Adversarial Robustness AAAI 2021 Learning by Turning: Neural Architecture Aware Optimisation ICML 2021 On Plasticity, Invariance, and Mutually Frozen Weights in Sequential Task Learning NeurIPS 2021 On the Periodic Behavior of Neural Network Training with Batch Normalization and Weight Decay NeurIPS 2021 Spherical Motion Dynamics: Learning Dynamics of Normalized Neural Network using SGD and Weight Decay NeurIPS 2021 Understanding Decoupled and Early Weight Decay AAAI 2021 Absum: Simple Regularization Method for Reducing Structural Sensitivity of Convolutional Neural Networks AAAI 2020 A Generalized Neural Tangent Kernel Analysis for Two-layer Neural Networks NeurIPS 2020 An Exponential Learning Rate Schedule for Deep Learning ICLR 2020 Delta-STN: Efficient Bilevel Optimization for Neural Networks using Structured Response Jacobians NeurIPS 2020 Fiedler Regularization: Learning Neural Networks with Graph Sparsity ICML 2020 Finite Versus Infinite Neural Networks: an Empirical Study NeurIPS 2020 Four Things Everyone Should Know to Improve Batch Normalization ICLR 2020 Meta-Learning with Adaptive Hyperparameters NeurIPS 2020 Neural Networks are Convex Regularizers: Exact Polynomial-time Convex Optimization Formulations for Two-layer Networks ICML 2020 Reconciling Modern Deep Learning with Traditional Optimization Analyses: The Intrinsic Learning Rate NeurIPS 2020 Residual Continual Learning AAAI 2020 Understanding Generalization in Recurrent Neural Networks ICLR 2020 Understanding the Disharmony between Weight Normalization Family and Weight Decay AAAI 2020 Decoupled Weight Decay Regularization ICLR 2019 Equi-normalization of Neural Networks ICLR 2019 On Implicit Filter Level Sparsity in Convolutional Neural Networks CVPR 2019 Piecewise Strong Convexity of Neural Networks NeurIPS 2019 Three Mechanisms of Weight Decay Regularization ICLR 2019 Time Matters in Regularizing Deep Networks: Weight Decay and Data Augmentation Affect Early Learning Dynamics, Matter Little Near Convergence NeurIPS 2019