Neural Network Optimization

PV-Tuning: Beyond Straight-Through Estimation for Extreme LLM Compression NeurIPS 2024 QLABGrad: A Hyperparameter-Free and Convergence-Guaranteed Scheme for Deep Learning AAAI 2024 QT-ViT: Improving Linear Attention in ViT with Quadratic Taylor Expansion NeurIPS 2024 Random Linear Projections Loss for Hyperplane-Based Optimization in Neural Networks UAI 2024 Reducing the Side-Effects of Oscillations in Training of Quantized YOLO Networks WACV 2024 Reparameterized Multi-Resolution Convolutions for Long Sequence Modelling NeurIPS 2024 Rethinking Weight Decay for Robust Fine-Tuning of Foundation Models NeurIPS 2024 Rewrite the Stars CVPR 2024 SAMPa: Sharpness-aware Minimization Parallelized NeurIPS 2024 SasWOT: Real-Time Semantic Segmentation Architecture Search WithOut Training AAAI 2024 Scalable Optimization in the Modular Norm NeurIPS 2024 Scaled Conjugate Gradient Method for Nonconvex Optimization in Deep Neural Networks JMLR 2024 Scaling Law for Time Series Forecasting NeurIPS 2024 Scaling Laws and Compute-Optimal Training Beyond Fixed Training Durations NeurIPS 2024 SDGMNet: Statistic-Based Dynamic Gradient Modulation for Local Descriptor Learning AAAI 2024 SDPose: Tokenized Pose Estimation via Circulation-Guide Self-Distillation CVPR 2024 Second-order forward-mode optimization of recurrent neural networks for neuroscience NeurIPS 2024 Separation and Bias of Deep Equilibrium Models on Expressivity and Learning Dynamics NeurIPS 2024 SGD vs GD: Rank Deficiency in Linear Networks NeurIPS 2024 Sharpened Lazy Incremental Quasi-Newton Method AISTATS 2024 Sharpness-Aware Minimization and the Edge of Stability JMLR 2024 Soft ascent-descent as a stable and flexible alternative to flooding NeurIPS 2024 Sorted LLaMA: Unlocking the Potential of Intermediate Layers of Large Language Models for Dynamic Inference EACL 2024 Sparse High Rank Adapters NeurIPS 2024 Sparse maximal update parameterization: A holistic approach to sparse training dynamics NeurIPS 2024