conftrace_

weight decay

50 papers

Explore in graph

Also known as

WD

Co-occurring keywords

neural network (6616) neural network optimization (1293) stochastic gradient descent (1091) convolutional neural network (4226) learning rate (192) deep network (152) gradient descent (1144) batch normalization (222) adversarial robustness (1341) optimization dynamics (10)

Papers

Global Optimality Beyond Two Layers: Training Deep ReLU Networks via Convex Programs ICML 2021
On Plasticity, Invariance, and Mutually Frozen Weights in Sequential Task Learning NIPS 2021
Spherical Motion Dynamics: Learning Dynamics of Normalized Neural Network using SGD and Weight Decay NIPS 2021
On the Periodic Behavior of Neural Network Training with Batch Normalization and Weight Decay NIPS 2021
Large Norms of CNN Layers Do Not Hurt Adversarial Robustness AAAI 2021
Adaptive Knowledge Driven Regularization for Deep Neural Networks AAAI 2021
Absum: Simple Regularization Method for Reducing Structural Sensitivity of Convolutional Neural Networks AAAI 2020
Reconciling Modern Deep Learning with Traditional Optimization Analyses: The Intrinsic Learning Rate NIPS 2020
Meta-Learning with Adaptive Hyperparameters NIPS 2020
Delta-STN: Efficient Bilevel Optimization for Neural Networks using Structured Response Jacobians NIPS 2020
Neural Networks are Convex Regularizers: Exact Polynomial-time Convex Optimization Formulations for Two-layer Networks ICML 2020
Understanding the Disharmony between Weight Normalization Family and Weight Decay AAAI 2020
Residual Continual Learning AAAI 2020
Fiedler Regularization: Learning Neural Networks with Graph Sparsity ICML 2020
Finite Versus Infinite Neural Networks: an Empirical Study NIPS 2020
A Generalized Neural Tangent Kernel Analysis for Two-layer Neural Networks NIPS 2020
Time Matters in Regularizing Deep Networks: Weight Decay and Data Augmentation Affect Early Learning Dynamics, Matter Little Near Convergence NIPS 2019
Piecewise Strong Convexity of Neural Networks NIPS 2019
On Implicit Filter Level Sparsity in Convolutional Neural Networks CVPR 2019
Norm matters: efficient and accurate normalization schemes in deep networks NIPS 2018
On the Power of Over-parametrization in Neural Networks with Quadratic Activation ICML 2018
Surprising properties of dropout in deep networks JMLR 2018
Learning Dynamics of Linear Denoising Autoencoders ICML 2018
Surprising properties of dropout in deep networks COLT 2017
The limits of squared Euclidean distance regularization NIPS 2014