Neural Network Optimization

Addax: Utilizing Zeroth-Order Gradients to Improve Memory Efficiency and Performance of SGD for Fine-Tuning Language Models ICLR 2025 A Method for Enhancing Generalization of Adam by Multiple Integrations AAAI 2025 A Mirror Descent Perspective of Smoothed Sign Descent UAI 2025 Analysis of Linear Mode Connectivity via Permutation-Based Weight Matching: With Insights into Other Permutation Search Methods ICLR 2025 A New Perspective on Shampoo's Preconditioner ICLR 2025 A New Random Reshuffling Method for Nonsmooth Nonconvex Finite-sum Optimization JMLR 2025 AniGrad: Anisotropic Gradient-Adaptive Sampling for 3D Reconstruction From Monocular Video CVPR 2025 A Proactive Reliability Metric for Detecting Failures in Language Model Training EMNLP 2025 A Semantic-Aware Layer-Freezing Approach to Computation-Efficient Fine-Tuning of Language Models ACL 2025 A Training-Free Length Extrapolation Approach for LLMs: Greedy Attention Logit Interpolation EMNLP 2025 Balcony: A Lightweight Approach to Dynamic Inference of Generative Language Models EMNLP 2025 Better Embeddings with Coupled Adam ACL 2025 Beyond Skip Connection: Pooling and Unpooling Design for Elimination Singularities AAAI 2025 COUNTDOWN: Contextually Sparse Activation Filtering Out Unnecessary Weights in Down Projection EMNLP 2025 Coupling-based Convergence Diagnostic and Stepsize Scheme for Stochastic Gradient Descent AAAI 2025 DAST: Difficulty-Adaptive Slow-Thinking for Large Reasoning Models EMNLP 2025 DeepLA-Net: Very Deep Local Aggregation Networks for Point Cloud Analysis CVPR 2025 DiG: Scalable and Efficient Diffusion Models with Gated Linear Attention CVPR 2025 Dynamic Algorithm Termination for Branch-and-Bound-based Neural Network Verification AAAI 2025 Dynamic Back-Substitution in Bound-Propagation-Based Neural Network Verification AAAI 2025 ECHO-LLaMA: Efficient Caching for High-Performance LLaMA Training EMNLP 2025 EcoTune: Token-Efficient Multi-Fidelity Hyperparameter Optimization for Large Language Model Inference EMNLP 2025 Efficient Layer-wise LLM Fine-tuning for Revision Intention Prediction EMNLP 2025 Efficient Many-Shot In-Context Learning with Dynamic Block-Sparse Attention ACL 2025 Efficient Training of Neural Fractional-Order Differential Equation via Adjoint Backpropagation AAAI 2025