deep learning optimization

ThunderKittens: Simple, Fast, and Adorable Kernels ICLR 2025 ADOPT: Modified Adam Can Converge with Any β_2 with the Optimal Rate NeurIPS 2024 Fixed-Weight Difference Target Propagation AAAI 2023 Revisit the Power of Vanilla Knowledge Distillation: from Small Scale to Large Scale NeurIPS 2023 AdaBlock: SGD with Practical Block Diagonal Matrix Adaptation for Deep Learning AISTATS 2022 A Momentumized, Adaptive, Dual Averaged Gradient Method JMLR 2022 To update or not to update? Neurons at equilibrium in deep models NeurIPS 2022 On the Acceleration of Deep Learning Model Parallelism With Staleness CVPR 2020 Provable Benefit of Orthogonal Initialization in Optimizing Deep Linear Networks ICLR 2020 A Theoretically Guaranteed Deep Optimization Framework for Robust Compressive Sensing MRI AAAI 2019 Generalized Batch Normalization: Towards Accelerating Deep Neural Networks AAAI 2019 On The Power of Curriculum Learning in Training Deep Networks ICML 2019 Ouroboros: On Accelerating Training of Transformer-Based Language Models NeurIPS 2019 BPGrad: Towards Global Optimality in Deep Learning via Branch and Pruning CVPR 2018