layer normalization

Improving Personalized Sentiment Representation with Knowledge-enhanced and Parameter-efficient Layer Normalization COLING 2024 MambaLLIE: Implicit Retinex-Aware Low Light Enhancement with Global-then-Local State Space NeurIPS 2024 MoEUT: Mixture-of-Experts Universal Transformers NeurIPS 2024 Normalization Layer Per-Example Gradients are Sufficient to Predict Gradient Noise Scale in Transformers NeurIPS 2024 On the Nonlinearity of Layer Normalization ICML 2024 On the Role of Attention Masks and LayerNorm in Transformers NeurIPS 2024 Reducing Fine-Tuning Memory Overhead by Approximate and Memory-Sharing Backpropagation ICML 2024 Strong Baselines for Parameter-Efficient Few-Shot Fine-Tuning AAAI 2024 StyleSinger: Style Transfer for Out-of-Domain Singing Voice Synthesis AAAI 2024 Transformer Fusion with Optimal Transport ICLR 2024 Transformers on Markov data: Constant depth suffices NeurIPS 2024 Tuning LayerNorm in Attention: Towards Efficient Multi-Modal LLM Finetuning ICLR 2024 Zipformer: A faster and better encoder for automatic speech recognition ICLR 2024 Achieve the Minimum Width of Neural Networks for Universal Approximation ICLR 2023 B2T Connection: Serving Stability and Performance in Deep Transformers ACL 2023 Brainformers: Trading Simplicity for Efficiency ICML 2023 Critical Initialization of Wide and Deep Neural Networks using Partial Jacobians: General Theory and Applications NeurIPS 2023 Deep Transformers without Shortcuts: Modifying Self-attention for Faithful Signal Propagation ICLR 2023 Exploring the Impact of Layer Normalization for Zero-shot Neural Machine Translation ACL 2023 LaPE: Layer-adaptive Position Embedding for Vision Transformers with Independent Layer Normalization ICCV 2023 On Sensitivity and Robustness of Normalization Schemes to Input Distribution Shifts in Automatic MR Image Diagnosis MIDL 2023 On the Expressivity Role of LayerNorm in Transformers’ Attention ACL 2023 On the impact of activation and normalization in obtaining isometric embeddings at initialization NeurIPS 2023 Pre-RMSNorm and Pre-CRMSNorm Transformers: Equivalent and Efficient Pre-LN Transformers NeurIPS 2023 Stabilizing Transformer Training by Preventing Attention Entropy Collapse ICML 2023