rank collapse

OSA: Echocardiography Video Segmentation via Orthogonalized State Update and Anatomical Prior-aware Feature Enhancement CVPR 2026 Lambda-Skip Connections: the architectural component that prevents Rank Collapse ICLR 2025 Mind the Gap: a Spectral Analysis of Rank Collapse and Signal Propagation in Attention Layers ICML 2025 Mitigating Plasticity Loss in Continual Reinforcement Learning by Reducing Churn ICML 2025 Exploiting the Social-Like Prior in Transformer for Visual Reasoning AAAI 2024 On the Role of Attention Masks and LayerNorm in Transformers NeurIPS 2024 PIDformer: Transformer Meets Control Theory ICML 2024 Self-attention Networks Localize When QK-eigenspectrum Concentrates ICML 2024 On the impact of activation and normalization in obtaining isometric embeddings at initialization NeurIPS 2023 Signal Propagation in Transformers: Theoretical Perspectives and the Role of Rank Collapse NeurIPS 2022 Attention is not all you need: pure attention loses rank doubly exponentially with depth ICML 2021 Implicit Under-Parameterization Inhibits Data-Efficient Deep Reinforcement Learning ICLR 2021 Batch normalization provably avoids ranks collapse for randomly initialised deep networks NeurIPS 2020