Normalization

On Separate Normalization in Self-supervised Transformers NeurIPS 2023 On skip connections and normalisation layers in deep optimisation NeurIPS 2023 On the Expressivity Role of LayerNorm in Transformers’ Attention ACL 2023 On the impact of activation and normalization in obtaining isometric embeddings at initialization NeurIPS 2023 Overcoming Recency Bias of Normalization Statistics in Continual Learning: Balance and Adaptation NeurIPS 2023 Over-Training with Mixup May Hurt Generalization ICLR 2023 Pre-RMSNorm and Pre-CRMSNorm Transformers: Equivalent and Efficient Pre-LN Transformers NeurIPS 2023 RegBN: Batch Normalization of Multimodal Data with Regularization NeurIPS 2023 REPAIR: REnormalizing Permuted Activations for Interpolation Repair ICLR 2023 Revisiting Structured Dropout ACML 2023 Robust Feature Decoupling in Voice Conversion by Using Locality-Based Instance Normalization Interspeech 2023 Robust One-Class Classification with Signed Distance Function using 1-Lipschitz Neural Networks ICML 2023 SEAT: Stable and Explainable Attention AAAI 2023 Selective Frequency Network for Image Restoration ICLR 2023 Statistically Valid Variable Importance Assessment through Conditional Permutations NeurIPS 2023 TANGOS: Regularizing Tabular Neural Networks through Gradient Orthogonalization and Specialization ICLR 2023 Tanh Works Better with Asymmetry NeurIPS 2023 The Crucial Role of Normalization in Sharpness-Aware Minimization NeurIPS 2023 The Expressive Power of Tuning Only the Normalization Layers COLT 2023 The Implicit Bias of Batch Normalization in Linear Models and Two-layer Linear Convolutional Neural Networks COLT 2023 Towards Efficient and Accurate Winograd Convolution via Full Quantization NeurIPS 2023 Towards Robust Object Detection Invariant to Real-World Domain Shifts ICLR 2023 Transition-constant Normalization for Image Enhancement NeurIPS 2023 TTN: A Domain-Shift Aware Batch Normalization in Test-Time Adaptation ICLR 2023 TWINS: A Fine-Tuning Framework for Improved Transferability of Adversarial Robustness and Generalization CVPR 2023