Theory

SIREN: Shaping Representations for Detecting Out-of-Distribution Objects NeurIPS 2022 Size and depth of monotone neural networks: interpolation and approximation NeurIPS 2022 Sketching based Representations for Robust Image Classification with Provable Guarantees NeurIPS 2022 Softmax Bottleneck Makes Language Models Unable to Represent Multi-mode Word Distributions ACL 2022 Sparse Double Descent: Where Network Pruning Aggravates Overfitting ICML 2022 Spectral Bias in Practice: The Role of Function Frequency in Generalization NeurIPS 2022 Spectral Bias Outside the Training Set for Deep Networks in the Kernel Regime NeurIPS 2022 Statistically Meaningful Approximation: a Case Study on Approximating Turing Machines with Transformers NeurIPS 2022 Surprising Instabilities in Training Deep Networks and a Theoretical Analysis NeurIPS 2022 Synergy and Symmetry in Deep Learning: Interactions between the Data, Model, and Inference Algorithm ICML 2022 The alignment property of SGD noise and how it helps select flat minima: A stability analysis NeurIPS 2022 The Architectural Bottleneck Principle EMNLP 2022 The Curse of Unrolling: Rate of Differentiating Through Optimization NeurIPS 2022 The Effect of Manifold Entanglement and Intrinsic Dimensionality on Learning AAAI 2022 The Effects of Invertibility on the Representational Complexity of Encoders in Variational Autoencoders ICLR 2022 The merged-staircase property: a necessary and nearly sufficient condition for SGD learning of sparse functions on two-layer neural networks COLT 2022 The Neural Covariance SDE: Shaped Infinite Depth-and-Width Networks at Initialization NeurIPS 2022 The Neural Race Reduction: Dynamics of Abstraction in Gated Networks ICML 2022 Theoretically Provable Spiking Neural Networks NeurIPS 2022 The Pitfalls of Regularization in Off-Policy TD Learning NeurIPS 2022 The price of ignorance: how much does it cost to forget noise structure in low-rank matrix estimation? NeurIPS 2022 The Sample Complexity of One-Hidden-Layer Neural Networks NeurIPS 2022 Three-stage Evolution and Fast Equilibrium for SGD with Non-degerate Critical Points ICML 2022 Topology-aware Generalization of Decentralized SGD ICML 2022 Toward Equation of Motion for Deep Neural Networks: Continuous-time Gradient Descent and Discretization Error Analysis NeurIPS 2022