Neural Network Optimization

Retroformer: Retrospective Large Language Agents with Policy Gradient Optimization ICLR 2024 Revisiting Adversarial Training at Scale CVPR 2024 Revisiting Catastrophic Forgetting in Large Language Model Tuning EMNLP 2024 Revisiting Scalable Hessian Diagonal Approximations for Applications in Reinforcement Learning ICML 2024 Revisiting the Noise Model of Stochastic Gradient Descent AISTATS 2024 Rewrite the Stars CVPR 2024 RingAttention with Blockwise Transformers for Near-Infinite Context ICLR 2024 Rotational Equilibrium: How Weight Decay Balances Learning Across Neural Networks ICML 2024 Ruler: A Model-Agnostic Method to Control Generated Length for Large Language Models EMNLP 2024 Same Task, More Tokens: the Impact of Input Length on the Reasoning Performance of Large Language Models ACL 2024 SAM-PARSER: Fine-Tuning SAM Efficiently by Parameter Space Reconstruction AAAI 2024 SAMPa: Sharpness-aware Minimization Parallelized NeurIPS 2024 Sample Complexity of Neural Policy Mirror Descent for Policy Optimization on Low-Dimensional Manifolds JMLR 2024 Scalable Optimization in the Modular Norm NeurIPS 2024 Scaled Conjugate Gradient Method for Nonconvex Optimization in Deep Neural Networks JMLR 2024 Scaling Convex Neural Networks with Burer-Monteiro Factorization ICLR 2024 Scaling Exponents Across Parameterizations and Optimizers ICML 2024 Scaling Laws and Compute-Optimal Training Beyond Fixed Training Durations NeurIPS 2024 Scaling Up to Excellence: Practicing Model Scaling for Photo-Realistic Image Restoration In the Wild CVPR 2024 SCA: Selective Compression Attention for Efficiently Extending the Context Window of Large Language Models EMNLP 2024 SDGMNet: Statistic-Based Dynamic Gradient Modulation for Local Descriptor Learning AAAI 2024 SDPose: Tokenized Pose Estimation via Circulation-Guide Self-Distillation CVPR 2024 Separation and Bias of Deep Equilibrium Models on Expressivity and Learning Dynamics NeurIPS 2024 Sequence Alignment-based Similarity Metric in Evolutionary Neural Architecture Search AutoML 2024 SGD Finds then Tunes Features in Two-Layer Neural Networks with near-Optimal Sample Complexity: A Case Study in the XOR problem ICLR 2024