conftrace_

weight normalization

26 papers

Explore in graph

Also known as

WN

Co-occurring keywords

neural network optimization (1293) batch normalization (222) neural network (6616) gradient descent (1144) layer normalization (65) graph neural network (3962) gradient explosion (8) residual network (211) weight decay (50) rademacher complexity (109)

Papers

Gradient Weight-normalized Low-rank Projection for Efficient LLM Training AAAI 2025
Scalable Optimization in the Modular Norm NIPS 2024
Optimized Potential Initialization for Low-Latency Spiking Neural Networks AAAI 2022
On the Stability and Scalability of Node Perturbation Learning NIPS 2022
Deep Hybrid Models for Out-of-Distribution Detection CVPR 2022
Constrained Weight Optimization for Learning Without Activation Normalization WACV 2021
Expressive 1-Lipschitz Neural Networks for Robust Multiple Graph Learning against Adversarial Attacks ICML 2021
Aligned Structured Sparsity Learning for Efficient Image Super-Resolution NIPS 2021
Towards a unified framework for fair and stable graph representation learning UAI 2021
Towards Stable and Robust AdderNets NIPS 2021
Optimization Theory for ReLU Neural Networks Trained with Normalization Layers ICML 2020
Implicit Regularization and Convergence for Weight Normalization NIPS 2020
New Interpretations of Normalization Methods in Deep Learning AAAI 2020
Understanding the Disharmony between Weight Normalization Family and Weight Decay AAAI 2020
Fine-grained Optimization of Deep Neural Networks NIPS 2019
How to Initialize your Network? Robust Initialization for WeightNorm & ResNets NIPS 2019
Normalization Helps Training of Quantized LSTM NIPS 2019
Inherent Weight Normalization in Stochastic Neural Networks NIPS 2019
Theoretical Investigation of Generalization Bound for Residual Networks IJCAI 2019
Improving Neural Language Models with Weight Norm Initialization and Regularization EMNLP 2018
Norm matters: efficient and accurate normalization schemes in deep networks NIPS 2018
Fine-tuning on Clean Data for End-to-End Speech Translation: FBK @ IWSLT 2018 EMNLP 2018
The RWTH Aachen University English-German and German-English Unsupervised Neural Machine Translation Systems for WMT 2018 EMNLP 2018
Understanding Weight Normalized Deep Neural Networks with Rectified Linear Units NIPS 2018
Gradient Descent Learns One-hidden-layer CNN: Don’t be Afraid of Spurious Local Minima ICML 2018