Surya Ganguli

An analytic theory of creativity in convolutional diffusion models ICML 2025 Features are fate: a theory of transfer learning in high-dimensional regression ICML 2025 Get rich quick: exact solutions reveal how unbalanced initializations promote rapid feature learning NeurIPS 2024 Disentanglement with Biological Constraints: A Theory of Functional Cell Types ICLR 2023 Information Geometry of the Retinal Representation Manifold NeurIPS 2023 Pretraining task diversity and the emergence of non-Bayesian in-context learning for regression NeurIPS 2023 Stochastic Collapse: How Gradient Noise Attracts SGD Dynamics Towards Simpler Subnetworks NeurIPS 2023 The Asymmetric Maximum Margin Bias of Quasi-Homogeneous Neural Networks ICLR 2023 Unmasking the Lottery Ticket Hypothesis: What's Encoded in a Winning Ticket's Mask? ICLR 2023 Beyond neural scaling laws: beating power law scaling via data pruning NeurIPS 2022 How many degrees of freedom do we need to train deep networks: a loss landscape perspective ICLR 2022 Lottery Tickets on a Data Diet: Finding Initializations with Sparse Trainable Networks NeurIPS 2022 MetaMorph: Learning Universal Controllers with Transformers ICLR 2022 A theory of high dimensional regression with arbitrary correlations between input features and target functions: sample complexity, multiple descent curves and a hierarchy of phase transitions ICML 2021 Deep Learning on a Data Diet: Finding Important Examples Early in Training NeurIPS 2021 Explaining heterogeneity in medial entorhinal cortex with task-driven neural networks NeurIPS 2021 Neural Mechanics: Symmetry and Broken Conservation Laws in Deep Learning Dynamics ICLR 2021 Understanding self-supervised learning dynamics without contrastive pairs ICML 2021 Deep learning versus kernel learning: an empirical study of loss landscape geometry and the time evolution of the Neural Tangent Kernel NeurIPS 2020 Identifying Learning Rules From Neural Network Observables NeurIPS 2020 Predictive coding in balanced neural networks with noise, chaos and delays NeurIPS 2020 Pruning neural networks without any data by iteratively conserving synaptic flow NeurIPS 2020 RNNs can generate bounded hierarchical languages with optimal memory EMNLP 2020 Two Routes to Scalable Credit Assignment without Weight Symmetry ICML 2020 An analytic theory of generalization dynamics and transfer learning in deep linear networks ICLR 2019 A unified theory for the origin of grid cells through the lens of pattern formation NeurIPS 2019 A Unified Theory of Early Visual Representations from Retina to Cortex through Anatomically Constrained Deep CNNs ICLR 2019 From deep learning to mechanistic understanding in neuroscience: the structure of retinal prediction NeurIPS 2019 Reverse engineering recurrent networks for sentiment classification reveals line attractor dynamics NeurIPS 2019 Universality and individuality in neural dynamics across large populations of recurrent networks NeurIPS 2019 Statistical mechanics of low-rank tensor decomposition NeurIPS 2018 Task-Driven Convolutional Recurrent Models of the Visual System NeurIPS 2018 The emergence of multiple retinal cell types through efficient coding of natural movies NeurIPS 2018 The emergence of spectral universality in deep networks AISTATS 2018 Continual Learning Through Synaptic Intelligence ICML 2017 On the Expressive Power of Deep Neural Networks ICML 2017 Resurrecting the sigmoid in deep learning through dynamical isometry: theory and practice NeurIPS 2017 Variational Walkback: Learning a Transition Operator as a Stochastic Recurrent Net NeurIPS 2017 An equivalence between high dimensional Bayes optimal inference and M-estimation NeurIPS 2016 Deep Learning Models of the Retinal Response to Natural Scenes NeurIPS 2016 Exponential expressivity in deep neural networks through transient chaos NeurIPS 2016 Deep Knowledge Tracing NeurIPS 2015 Deep Unsupervised Learning using Nonequilibrium Thermodynamics ICML 2015 Fast large-scale optimization by unifying stochastic gradient and quasi-Newton methods ICML 2014 Identifying and attacking the saddle point problem in high-dimensional non-convex optimization NeurIPS 2014 A memory frontier for complex synapses NeurIPS 2013 Short-term memory in neuronal networks through dynamical compressed sensing NeurIPS 2010