neural network optimization

Adaptive Depth Networks with Skippable Sub-Paths NeurIPS 2024 A distributional simplicity bias in the learning dynamics of transformers NeurIPS 2024 ADOPT: Modified Adam Can Converge with Any β_2 with the Optimal Rate NeurIPS 2024 Advancing Training Efficiency of Deep Spiking Neural Networks through Rate-based Backpropagation NeurIPS 2024 AffineQuant: Affine Transformation Quantization for Large Language Models ICLR 2024 AFLoRA: Adaptive Freezing of Low Rank Adaptation in Parameter Efficient Fine-Tuning of Large Models ACL 2024 A Layer-Wise Natural Gradient Optimizer for Training Deep Neural Networks NeurIPS 2024 Algorithmic progress in language models NeurIPS 2024 Aligning Logits Generatively for Principled Black-Box Knowledge Distillation CVPR 2024 AlphaLoRA: Assigning LoRA Experts Based on Layer Training Quality EMNLP 2024 Analyzing and Improving the Training Dynamics of Diffusion Models CVPR 2024 Analyzing Generalization in Policy Networks: A Case Study with the Double-Integrator System AAAI 2024 Analyzing & Reducing the Need for Learning Rate Warmup in GPT Training NeurIPS 2024 An In-depth Investigation of Sparse Rate Reduction in Transformer-like Models NeurIPS 2024 AntLM: Bridging Causal and Masked Language Models CoNLL 2024 A PDE-based Explanation of Extreme Numerical Sensitivities and Edge of Stability in Training Neural Networks JMLR 2024 A provable control of sensitivity of neural networks through a direct parameterization of the overall bi-Lipschitzness NeurIPS 2024 A Single Linear Layer Yields Task-Adapted Low-Rank Matrices COLING 2024 A Single-Step, Sharpness-Aware Minimization is All You Need to Achieve Efficient and Accurate Sparse Training NeurIPS 2024 A Structure-Aware Framework for Learning Device Placements on Computation Graphs NeurIPS 2024 A Study of Plasticity Loss in On-Policy Deep Reinforcement Learning NeurIPS 2024 Attribution Patching Outperforms Automated Circuit Discovery EMNLP 2024 A two-scale Complexity Measure for Deep Learning Models NeurIPS 2024 AutoAugment Is What You Need: Enhancing Rule-based Augmentation Methods in Low-resource Regimes EACL 2024 BadRL: Sparse Targeted Backdoor Attack against Reinforcement Learning AAAI 2024