Neural Network Optimization

Over-parameterized Model Optimization with Polyak-{\L}ojasiewicz Condition ICLR 2023 PAC-tuning: Fine-tuning Pre-trained Language Models with PAC-driven Perturbed Gradient Descent EMNLP 2023 PA&DA: Jointly Sampling Path and Data for Consistent NAS CVPR 2023 Pareto Frontiers in Deep Feature Learning: Data, Compute, Width, and Luck NeurIPS 2023 Path Regularization: A Convexity and Sparsity Inducing Regularization for Parallel ReLU Networks NeurIPS 2023 PDE-Refiner: Achieving Accurate Long Rollouts with Neural PDE Solvers NeurIPS 2023 Perturbation Analysis of Neural Collapse ICML 2023 PGrad: Learning Principal Gradients For Domain Generalization ICLR 2023 Phase diagram of early training dynamics in deep neural networks: effect of the learning rate, depth, and width NeurIPS 2023 Phase-shifted adversarial training UAI 2023 Physics-Informed Model-Based Reinforcement Learning L4DC 2023 PLASTIC: Improving Input and Label Plasticity for Sample Efficient Reinforcement Learning NeurIPS 2023 Plateau in Monotonic Linear Interpolation --- A "Biased" View of Loss Landscape for Deep Networks ICLR 2023 Polarity Is All You Need to Learn and Transfer Faster ICML 2023 PPG Reloaded: An Empirical Study on What Matters in Phasic Policy Gradient ICML 2023 Practical and Matching Gradient Variance Bounds for Black-Box Variational Bayesian Inference ICML 2023 Practical Sharpness-Aware Minimization Cannot Converge All the Way to Optima NeurIPS 2023 Preconditioned Gradient Descent for Overparameterized Nonconvex Burer--Monteiro Factorization with Global Optimality Certification JMLR 2023 Prediction and Calibration: Complex Reasoning over Knowledge Graph with Bi-directional Directed Acyclic Graph Neural Network ACL 2023 Prefix Propagation: Parameter-Efficient Tuning for Long Sequences ACL 2023 Pre-RMSNorm and Pre-CRMSNorm Transformers: Equivalent and Efficient Pre-LN Transformers NeurIPS 2023 Pretraining Without Attention EMNLP 2023 Principled Weight Initialisation for Input-Convex Neural Networks NeurIPS 2023 Provable Advantage of Curriculum Learning on Parity Targets with Mixed Inputs NeurIPS 2023 PTP: Boosting Stability and Performance of Prompt Tuning with Perturbation-Based Regularizer EMNLP 2023