Neural Network Optimization

Expanding Sparse Tuning for Low Memory Usage NeurIPS 2024 Explicit Eigenvalue Regularization Improves Sharpness-Aware Minimization NeurIPS 2024 Faster Diffusion: Rethinking the Role of the Encoder for Diffusion Model Inference NeurIPS 2024 Faster Neighborhood Attention: Reducing the O(n^2) Cost of Self Attention at the Threadblock Level NeurIPS 2024 Fast Forwarding Low-Rank Training EMNLP 2024 Finding Lottery Tickets in Vision Models via Data-driven Spectral Foresight Pruning CVPR 2024 FlashAttention-3: Fast and Accurate Attention with Asynchrony and Low-precision NeurIPS 2024 From Activation to Initialization: Scaling Insights for Optimizing Neural Fields CVPR 2024 Get rich quick: exact solutions reveal how unbalanced initializations promote rapid feature learning NeurIPS 2024 Great Minds Think Alike: The Universal Convergence Trend of Input Salience NeurIPS 2024 How does Gradient Descent Learn Features --- A Local Analysis for Regularized Two-Layer Neural Networks NeurIPS 2024 Hybrid-Supervised Dual-Search: Leveraging Automatic Learning for Loss-Free Multi-Exposure Image Fusion AAAI 2024 Identifying Equivalent Training Dynamics NeurIPS 2024 Identifying Spurious Biases Early in Training through the Lens of Simplicity Bias AISTATS 2024 Implicit Bias in Noisy-SGD: With Applications to Differentially Private Training AISTATS 2024 Implicit Regularization of Sharpness-Aware Minimization for Scale-Invariant Problems NeurIPS 2024 Improving Equivariant Model Training via Constraint Relaxation NeurIPS 2024 Improving Training Efficiency of Diffusion Models via Multi-Stage Framework and Tailored Multi-Decoder Architecture CVPR 2024 Increasing Biases Can Be More Efficient Than Increasing Weights WACV 2024 Initialization of Large Language Models via Reparameterization to Mitigate Loss Spikes EMNLP 2024 In Search of a Data Transformation That Accelerates Neural Field Training CVPR 2024 Instance-Aware Group Quantization for Vision Transformers CVPR 2024 Investigation into Training Dynamics of Learned Optimizers (Student Abstract) AAAI 2024 Kronecker-Factored Approximate Curvature for Physics-Informed Neural Networks NeurIPS 2024 Large Stepsize Gradient Descent for Non-Homogeneous Two-Layer Networks: Margin Improvement and Fast Optimization NeurIPS 2024