Tri Dao

Ladder-Residual: Parallelism-Aware Architecture for Accelerating Large Model Inference with Communication Overlapping ICML 2025 Long-Context State-Space Video World Models ICCV 2025 BitDelta: Your Fine-Tune May Only Be Worth One Bit NeurIPS 2024 Caduceus: Bi-Directional Equivariant Long-Range DNA Sequence Modeling ICML 2024 FlashAttention-2: Faster Attention with Better Parallelism and Work Partitioning ICLR 2024 FlashAttention-3: Fast and Accurate Attention with Asynchrony and Low-precision NeurIPS 2024 Hydra: Bidirectional State Space Models Through Generalized Matrix Mixers NeurIPS 2024 Medusa: Simple LLM Inference Acceleration Framework with Multiple Decoding Heads ICML 2024 RedPajama: an Open Dataset for Training Large Language Models NeurIPS 2024 The Mamba in the Llama: Distilling and Accelerating Hybrid Models NeurIPS 2024 Transformers are SSMs: Generalized Models and Efficient Algorithms Through Structured State Space Duality ICML 2024 Deja Vu: Contextual Sparsity for Efficient LLMs at Inference Time ICML 2023 Effectively Modeling Time Series with Simple Discrete State Spaces ICLR 2023 Hungry Hungry Hippos: Towards Language Modeling with State Space Models ICLR 2023 Hyena Hierarchy: Towards Larger Convolutional Language Models ICML 2023 Simple Hardware-Efficient Long Convolutions for Sequence Modeling ICML 2023 ButterflyFlow: Building Invertible Layers with Butterfly Matrices ICML 2022 Decentralized Training of Foundation Models in Heterogeneous Environments NeurIPS 2022 Fine-tuning Language Models over Slow Networks using Activation Quantization with Guarantees NeurIPS 2022 FlashAttention: Fast and Memory-Efficient Exact Attention with IO-Awareness NeurIPS 2022 Monarch: Expressive Structured Matrices for Efficient and Accurate Training ICML 2022 Pixelated Butterfly: Simple and Efficient Sparse training for Neural Network Models ICLR 2022 S4ND: Modeling Images and Videos as Multidimensional Signals with State Spaces NeurIPS 2022 Transform Once: Efficient Operator Learning in Frequency Domain NeurIPS 2022 Catformer: Designing Stable Transformers via Sensitivity Analysis ICML 2021 Combining Recurrent, Convolutional, and Continuous-time Models with Linear State Space Layers NeurIPS 2021 Knowledge Distillation as Semiparametric Inference ICLR 2021 MONGOOSE: A Learnable LSH Framework for Efficient Neural Network Training ICLR 2021 Rethinking Neural Operations for Diverse Tasks NeurIPS 2021 Scatterbrain: Unifying Sparse and Low-rank Attention NeurIPS 2021 HiPPO: Recurrent Memory with Optimal Polynomial Projections NeurIPS 2020 Kaleidoscope: An Efficient, Learnable Representation For All Structured Linear Maps ICLR 2020 Adaptive Hashing for Model Counting UAI 2019 A Kernel Theory of Modern Data Augmentation ICML 2019 Approximating the Permanent by Sampling from Adaptive Partitions NeurIPS 2019 Learning Fast Algorithms for Linear Transforms Using Butterfly Factorizations ICML 2019 Low-Precision Random Fourier Features for Memory-constrained Kernel Approximation AISTATS 2019 On the Downstream Performance of Compressed Word Embeddings NeurIPS 2019 Learning Compressed Transforms with Low Displacement Rank NeurIPS 2018 Gaussian Quadrature for Kernel Features NeurIPS 2017