Albert Gu

On the Benefits of Memory for Modeling Time-Dependent PDEs ICLR 2025 Towards Codec-LM Co-design for Neural Codec Language Models NAACL 2025 Understanding and Improving Length Generalization in Recurrent Models ICML 2025 Understanding the Skill Gap in Recurrent Language Models: The Role of the Gather-and-Aggregate Mechanism ICML 2025 Caduceus: Bi-Directional Equivariant Long-Range DNA Sequence Modeling ICML 2024 Hydra: Bidirectional State Space Models Through Generalized Matrix Mixers NeurIPS 2024 Transformers are SSMs: Generalized Models and Efficient Algorithms Through Structured State Space Duality ICML 2024 Transformers to SSMs: Distilling Quadratic Knowledge to Subquadratic Models NeurIPS 2024 How to Train your HIPPO: State Space Models with Generalized Orthogonal Basis Projections ICLR 2023 Modelling Long Range Dependencies in ND: From Task-Specific to a General Purpose CNN ICLR 2023 Pretraining Without Attention EMNLP 2023 Resurrecting Recurrent Neural Networks for Long Sequences ICML 2023 Structured State Space Models for In-Context Reinforcement Learning NeurIPS 2023 Diagonal State Spaces are as Effective as Structured State Spaces NeurIPS 2022 Efficiently Modeling Long Sequences with Structured State Spaces ICLR 2022 It’s Raw! Audio Generation with State-Space Models ICML 2022 On the Parameterization and Initialization of Diagonal State Space Models NeurIPS 2022 S4ND: Modeling Images and Videos as Multidimensional Signals with State Spaces NeurIPS 2022 Catformer: Designing Stable Transformers via Sensitivity Analysis ICML 2021 Combining Recurrent, Convolutional, and Continuous-time Models with Linear State Space Layers NeurIPS 2021 HoroPCA: Hyperbolic Dimensionality Reduction via Horospherical Projections ICML 2021 Model Patching: Closing the Subgroup Performance Gap with Data Augmentation ICLR 2021 From Trees to Continuous Embeddings and Back: Hyperbolic Hierarchical Clustering NeurIPS 2020 HiPPO: Recurrent Memory with Optimal Polynomial Projections NeurIPS 2020 Improving the Gating Mechanism of Recurrent Neural Networks ICML 2020 Kaleidoscope: An Efficient, Learnable Representation For All Structured Linear Maps ICLR 2020 No Subclass Left Behind: Fine-Grained Robustness in Coarse-Grained Classification Problems NeurIPS 2020 A Kernel Theory of Modern Data Augmentation ICML 2019 Learning Fast Algorithms for Linear Transforms Using Butterfly Factorizations ICML 2019 Learning Mixed-Curvature Representations in Product Spaces ICLR 2019 Learning Compressed Transforms with Low Displacement Rank NeurIPS 2018 Representation Tradeoffs for Hyperbolic Embeddings ICML 2018