Neural Network Optimization

Vision Mamba Mender NeurIPS 2024 VNN: Verification-Friendly Neural Networks with Hard Robustness Guarantees ICML 2024 Weight-Entanglement Meets Gradient-Based Neural Architecture Search AutoML 2024 What Makes a Good Prune? Maximal Unstructured Pruning for Maximal Cosine Similarity ICLR 2024 When to Grow? A Fitting Risk-Aware Policy for Layer Growing in Deep Neural Networks AAAI 2024 Where Do Large Learning Rates Lead Us? NeurIPS 2024 Why Do We Need Weight Decay in Modern Deep Learning? NeurIPS 2024 Why Transformers Need Adam: A Hessian Perspective NeurIPS 2024 Why Warmup the Learning Rate? Underlying Mechanisms and Improvements NeurIPS 2024 Wino Vidi Vici: Conquering Numerical Instability of 8-Bit Winograd Convolution for Accurate Inference Acceleration on Edge WACV 2024 wTIMIT2mix: A Cocktail Party Mixtures Database to Study Target Speaker Extraction for Normal and Whispered Speech Interspeech 2024 ZeroRF: Fast Sparse View 360deg Reconstruction with Zero Pretraining CVPR 2024 3D Human Pose Estimation With Spatio-Temporal Criss-Cross Attention CVPR 2023 Abide by the law and follow the flow: conservation laws for gradient flows NeurIPS 2023 Absolute Position Embedding Learns Sinusoid-like Waves for Attention Based on Relative Position EMNLP 2023 Accelerated Quasi-Newton Proximal Extragradient: Faster Rate for Smooth Convex Optimization NeurIPS 2023 Accelerated Stochastic Optimization Methods under Quasar-convexity ICML 2023 Accelerated Training via Incrementally Growing Neural Networks using Variance Transfer and Learning Rate Adaptation NeurIPS 2023 Acceleration of Large Transformer Model Training by Sensitivity-Based Layer Dropping AAAI 2023 Accumulated Trivial Attention Matters in Vision Transformers on Small Datasets WACV 2023 Accurate Neural Training with 4-bit Matrix Multiplications at Standard Formats ICLR 2023 Achieve the Minimum Width of Neural Networks for Universal Approximation ICLR 2023 Achieving O(ε^-1.5) Complexity in Hessian/Jacobian-free Stochastic Bilevel Optimization NeurIPS 2023 A Computationally Efficient Sparsified Online Newton Method NeurIPS 2023 Adaptive Optimization in the ∞-Width Limit ICLR 2023