Neural Network Optimization

Neural Discovery of Permutation Subgroups AISTATS 2023 Neural Fourier Filter Bank CVPR 2023 Neural Networks Efficiently Learn Low-Dimensional Representations with SGD ICLR 2023 Neural networks trained with SGD learn distributions of increasing complexity ICML 2023 Neural tangent kernel at initialization: linear width suffices UAI 2023 Neural (Tangent Kernel) Collapse NeurIPS 2023 Neural Wave Machines: Learning Spatiotemporally Structured Representations with Locally Coupled Oscillatory Recurrent Neural Networks ICML 2023 NIRVANA: Neural Implicit Representations of Videos With Adaptive Networks and Autoregressive Patch-Wise Modeling CVPR 2023 “No Free Lunch” in Neural Architectures? A Joint Analysis of Expressivity, Convergence, and Generalization AutoML 2023 Non-autoregressive Streaming Transformer for Simultaneous Translation EMNLP 2023 Nonlinear Advantage: Trained Networks Might Not Be As Complex as You Think ICML 2023 Normalization Layers Are All That Sharpness-Aware Minimization Needs NeurIPS 2023 No Train No Gain: Revisiting Efficient Training Algorithms For Transformer-based Language Models NeurIPS 2023 NTK-approximating MLP Fusion for Efficient Language Model Fine-tuning ICML 2023 NTK-SAP: Improving neural network pruning by aligning training dynamics ICLR 2023 Nyström Method for Accurate and Scalable Implicit Differentiation AISTATS 2023 Offline Reinforcement Learning with Differentiable Function Approximation is Provably Efficient ICLR 2023 On Enhancing Fine-Tuning for Pre-trained Language Models EMNLP 2023 One Wide Feedforward Is All You Need EMNLP 2023 On Learning Rates and Schrödinger Operators JMLR 2023 Online Stochastic Gradient Descent with Arbitrary Initialization Solves Non-smooth, Non-convex Phase Retrieval JMLR 2023 On Over-Squashing in Message Passing Neural Networks: The Impact of Width, Depth, and Topology ICML 2023 On quantum backpropagation, information reuse, and cheating measurement collapse NeurIPS 2023 On Single-Index Models beyond Gaussian Data NeurIPS 2023 On skip connections and normalisation layers in deep optimisation NeurIPS 2023