layer normalization

The Expressive Power of Tuning Only the Normalization Layers COLT 2023 Understanding, Predicting and Better Resolving Q-Value Divergence in Offline-RL NeurIPS 2023 An Empirical Study on the Transferability of Transformer Modules in Parameter-efficient Fine-tuning EMNLP 2022 Dropout Q-Functions for Doubly Efficient Reinforcement Learning ICLR 2022 Dynamic Token Normalization improves Vision Transformers ICLR 2022 Measuring the Mixing of Contextual Information in the Transformer EMNLP 2022 Outlier Suppression: Pushing the Limit of Low-bit Transformer Language Models NeurIPS 2022 Overcoming a Theoretical Limitation of Self-Attention ACL 2022 Revisiting Over-smoothing in BERT from the Perspective of Graph ICLR 2022 Understanding the Failure of Batch Normalization for Transformers in NLP NeurIPS 2022 Understanding the Generalization Benefit of Normalization Layers: Sharpness Reduction NeurIPS 2022 Unsupervised Photo-to-Caricature Generation with Adaptive Select Layer-Instance Normalization and Semi-cycle Consistency ACML 2022 XAI for Transformers: Better Explanations through Conservative Propagation ICML 2022 Beyond BatchNorm: Towards a Unified Understanding of Normalization in Deep Learning NeurIPS 2021 GraphNorm: A Principled Approach to Accelerating Graph Neural Network Training ICML 2021 Incorporating Residual and Normalization Layers into Analysis of Masked Language Models EMNLP 2021 Meta-StyleSpeech : Multi-Speaker Adaptive Text-to-Speech Generation ICML 2021 Proxy-Normalizing Activations to Match Batch Normalization while Removing Batch Dependence NeurIPS 2021 Residual Shuffle-Exchange Networks for Fast Processing of Long Sequences AAAI 2021 Sparse Attention with Linear Units EMNLP 2021 Capsules with Inverted Dot-Product Attention Routing ICLR 2020 Improving Transformer Optimization Through Better Initialization ICML 2020 Lipschitz Constrained Parameter Initialization for Deep Transformers ACL 2020 New Interpretations of Normalization Methods in Deep Learning AAAI 2020 On Layer Normalization in the Transformer Architecture ICML 2020