conftrace_

layer normalization

65 papers

Explore in graph

Also known as

LN LAYERNORM

Co-occurring keywords

batch normalization (222) neural network optimization (1293) neural network (6616) group normalization (13) transformer architecture (1560) deep neural network (1803) attention mechanism (4001) residual connection (55) transformer model (1996) model optimization (75)

Papers

Brainformers: Trading Simplicity for Efficiency ICML 2023
On the Expressivity Role of LayerNorm in Transformers’ Attention ACL 2023
On Sensitivity and Robustness of Normalization Schemes to Input Distribution Shifts in Automatic MR Image Diagnosis MIDL 2023
The Expressive Power of Tuning Only the Normalization Layers COLT 2023
LaPE: Layer-adaptive Position Embedding for Vision Transformers with Independent Layer Normalization ICCV 2023
B2T Connection: Serving Stability and Performance in Deep Transformers ACL 2023
Exploring the Impact of Layer Normalization for Zero-shot Neural Machine Translation ACL 2023
Understanding the Failure of Batch Normalization for Transformers in NLP NIPS 2022
Outlier Suppression: Pushing the Limit of Low-bit Transformer Language Models NIPS 2022
Overcoming a Theoretical Limitation of Self-Attention ACL 2022
Measuring the Mixing of Contextual Information in the Transformer EMNLP 2022
XAI for Transformers: Better Explanations through Conservative Propagation ICML 2022
Unsupervised Photo-to-Caricature Generation with Adaptive Select Layer-Instance Normalization and Semi-cycle Consistency ACML 2022
An Empirical Study on the Transferability of Transformer Modules in Parameter-efficient Fine-tuning EMNLP 2022
Understanding the Generalization Benefit of Normalization Layers: Sharpness Reduction NIPS 2022
Proxy-Normalizing Activations to Match Batch Normalization while Removing Batch Dependence NIPS 2021
Sparse Attention with Linear Units EMNLP 2021
Incorporating Residual and Normalization Layers into Analysis of Masked Language Models EMNLP 2021
Meta-StyleSpeech : Multi-Speaker Adaptive Text-to-Speech Generation ICML 2021
GraphNorm: A Principled Approach to Accelerating Graph Neural Network Training ICML 2021
Residual Shuffle-Exchange Networks for Fast Processing of Long Sequences AAAI 2021
Beyond BatchNorm: Towards a Unified Understanding of Normalization in Deep Learning NIPS 2021
New Interpretations of Normalization Methods in Deep Learning AAAI 2020
Rethinking Skip Connection with Layer Normalization COLING 2020
On Layer Normalization in the Transformer Architecture ICML 2020