Surya Ganguli

47 papers · 2010–2025 · 5 conferences · across top CS/AI conferences

Achievements

+14 more ↓

🧭 Keyword Pioneer 🌍 Conference Polyglot (5) 🌉 Interdisciplinary Bridge 🗺️ Taxonomy Completionist (12) 🏃 Academic Marathon (15)

🗺️ Taxonomy Completionist (12) 🏃 Academic Marathon (15) 🧭 Keyword Pioneer 🌟 Keyword Trendsetter Combo (4) 🏠 Conference Loyalist (28) 🏆 Keyword Champion 👑 Triple Crown 🔬 Deep Specialist (18) ⚡ Prolific Year (6) 🗃️ Keyword Collector (192) 💎 Century Club (47) ❓ The Questioner 🔥 Unstoppable (13) 📈 Trend Setter

Conferences

NIPS (28) ICML (9) ICLR (8) AISTATS (1) EMNLP (1)

Top co-authors

Daniel Kunin (6) Aran Nayebi (6) Jascha Sohl-dickstein (5) Mansheej Paul (5) Niru Maheswaranathan (5) Gintare Karolina Dziugaite (4) Feng Chen (4) Ben Poole (4) Ben Sorscher (3) David Sussillo (3)

Research topics

Education (1)

Keywords

neural network (6) recurrent neural network (6) dynamical system (4) deep neural network (4) phase transition (3) convolutional neural network (3) training dynamics (2) stochastic gradient descent (2) sparse training (2) representation learning (2) neural response (2) synaptic memory (2) feature learning (2) lottery ticket hypothesis (2) sample complexity (2) neural encoding (2) gradient descent (2) neural dynamics (2) loss landscape (2) generative model (2)

Papers

Features are fate: a theory of transfer learning in high-dimensional regression ICML 2025 An analytic theory of creativity in convolutional diffusion models ICML 2025 Get rich quick: exact solutions reveal how unbalanced initializations promote rapid feature learning NIPS 2024 Pretraining task diversity and the emergence of non-Bayesian in-context learning for regression NIPS 2023 Stochastic Collapse: How Gradient Noise Attracts SGD Dynamics Towards Simpler Subnetworks NIPS 2023 Unmasking the Lottery Ticket Hypothesis: What's Encoded in a Winning Ticket's Mask? ICLR 2023 Disentanglement with Biological Constraints: A Theory of Functional Cell Types ICLR 2023 The Asymmetric Maximum Margin Bias of Quasi-Homogeneous Neural Networks ICLR 2023 Information Geometry of the Retinal Representation Manifold NIPS 2023 MetaMorph: Learning Universal Controllers with Transformers ICLR 2022 How many degrees of freedom do we need to train deep networks: a loss landscape perspective ICLR 2022 Lottery Tickets on a Data Diet: Finding Initializations with Sparse Trainable Networks NIPS 2022 Beyond neural scaling laws: beating power law scaling via data pruning NIPS 2022 A theory of high dimensional regression with arbitrary correlations between input features and target functions: sample complexity, multiple descent curves and a hierarchy of phase transitions ICML 2021 Deep Learning on a Data Diet: Finding Important Examples Early in Training NIPS 2021 Understanding self-supervised learning dynamics without contrastive pairs ICML 2021 Explaining heterogeneity in medial entorhinal cortex with task-driven neural networks NIPS 2021 Neural Mechanics: Symmetry and Broken Conservation Laws in Deep Learning Dynamics ICLR 2021 Pruning neural networks without any data by iteratively conserving synaptic flow NIPS 2020 Identifying Learning Rules From Neural Network Observables NIPS 2020 Deep learning versus kernel learning: an empirical study of loss landscape geometry and the time evolution of the Neural Tangent Kernel NIPS 2020 Predictive coding in balanced neural networks with noise, chaos and delays NIPS 2020 RNNs can generate bounded hierarchical languages with optimal memory EMNLP 2020 Two Routes to Scalable Credit Assignment without Weight Symmetry ICML 2020 Reverse engineering recurrent networks for sentiment classification reveals line attractor dynamics NIPS 2019 A unified theory for the origin of grid cells through the lens of pattern formation NIPS 2019 Universality and individuality in neural dynamics across large populations of recurrent networks NIPS 2019 A Unified Theory of Early Visual Representations from Retina to Cortex through Anatomically Constrained Deep CNNs ICLR 2019 An analytic theory of generalization dynamics and transfer learning in deep linear networks ICLR 2019 From deep learning to mechanistic understanding in neuroscience: the structure of retinal prediction NIPS 2019 The emergence of spectral universality in deep networks AISTATS 2018 Task-Driven Convolutional Recurrent Models of the Visual System NIPS 2018 The emergence of multiple retinal cell types through efficient coding of natural movies NIPS 2018 Statistical mechanics of low-rank tensor decomposition NIPS 2018 Resurrecting the sigmoid in deep learning through dynamical isometry: theory and practice NIPS 2017 Continual Learning Through Synaptic Intelligence ICML 2017 On the Expressive Power of Deep Neural Networks ICML 2017 Variational Walkback: Learning a Transition Operator as a Stochastic Recurrent Net NIPS 2017 Deep Learning Models of the Retinal Response to Natural Scenes NIPS 2016 An equivalence between high dimensional Bayes optimal inference and M-estimation NIPS 2016 Exponential expressivity in deep neural networks through transient chaos NIPS 2016 Deep Unsupervised Learning using Nonequilibrium Thermodynamics ICML 2015 Deep Knowledge Tracing NIPS 2015 Identifying and attacking the saddle point problem in high-dimensional non-convex optimization NIPS 2014 Fast large-scale optimization by unifying stochastic gradient and quasi-Newton methods ICML 2014 A memory frontier for complex synapses NIPS 2013 Short-term memory in neuronal networks through dynamical compressed sensing NIPS 2010