Neural Network Optimization

Not Everything is All You Need: Toward Low-Redundant Optimization for Large Language Model Alignment EMNLP 2024 No Wrong Turns: The Simple Geometry Of Neural Networks Optimization Paths ICML 2024 Object segmentation from common fate: Motion energy processing enables human-like zero-shot generalization to random dot stimuli NeurIPS 2024 Once for Both: Single Stage of Importance and Sparsity Search for Vision Transformer Compression CVPR 2024 On Convergence of Adam for Stochastic Optimization under Relaxed Assumptions NeurIPS 2024 On Convergence of Incremental Gradient for Non-convex Smooth Functions ICML 2024 On Double Descent in Reinforcement Learning with LSTD and Random Features ICLR 2024 OneBit: Towards Extremely Low-bit Large Language Models NeurIPS 2024 One Forward is Enough for Neural Network Training via Likelihood Ratio Method ICLR 2024 One-Layer Transformer Provably Learns One-Nearest Neighbor In Context NeurIPS 2024 One Step of Gradient Descent is Provably the Optimal In-Context Learner with One Layer of Linear Self-Attention ICLR 2024 On Feature Learning in Structured State Space Models NeurIPS 2024 On Improving Error Resilience of Neural End-to-End Speech Coders Interspeech 2024 Online Learning and Information Exponents: The Importance of Batch size & Time/Complexity Tradeoffs ICML 2024 On Mesa-Optimization in Autoregressively Trained Transformers: Emergence and Capability NeurIPS 2024 On Spectral Properties of Gradient-based Explanation Methods ECCV 2024 On the Computational and Statistical Complexity of Over-parameterized Matrix Sensing JMLR 2024 On the Duality Between Sharpness-Aware Minimization and Adversarial Training ICML 2024 On the Effect of Initialization: The Scaling Path of 2-Layer Neural Networks JMLR 2024 On-the-fly Denoising for Data Augmentation in Natural Language Understanding EACL 2024 On the Generalization of Stochastic Gradient Descent with Momentum JMLR 2024 On the Hardness of Probabilistic Neurosymbolic Learning ICML 2024 On the Hyperparameters in Stochastic Gradient Descent with Momentum JMLR 2024 On the Impact of Overparameterization on the Training of a Shallow Neural Network in High Dimensions AISTATS 2024 On the Impacts of the Random Initialization in the Neural Tangent Kernel Theory NeurIPS 2024