Jascha Sohl-dickstein

44 papers · 2012–2024 · 5 conferences · across top CS/AI conferences

Achievements

+14 more ↓

🗺️ Taxonomy Completionist (14) 🧭 Keyword Pioneer 🌉 Interdisciplinary Bridge 🌈 Renaissance Researcher (6) 🌍 Conference Polyglot (5)

🌉 Interdisciplinary Bridge 🐝 Cross-Pollinator (12) 🗺️ Taxonomy Completionist (14) 🌟 Keyword Trendsetter Combo (5) 🤝 Dynamic Duo (11) 👑 Triple Crown 🔬 Deep Specialist (12) 🏆 Keyword Champion (2) 📈 Trend Setter 🚀 Conference Pioneer ⚡ Prolific Year (5) 🗃️ Keyword Collector (146) 💎 Century Club (44) 🔥 Unstoppable (11)

Conferences

ICML (18) NIPS (14) ICLR (10) IJCAI (1) JMLR (1)

Top co-authors

Roman Novak (11) Jeffrey Pennington (10) Jaehoon Lee (8) Luke Metz (8) Lechao Xiao (7) Niru Maheswaranathan (6) Yasaman Bahri (6) Surya Ganguli (5) Ben Poole (4) Jiri Hron (4)

Research topics

Education (1)

Keywords

learned optimizer (5) neural network optimization (5) neural network (4) neural tangent kernel (3) markov chain monte carlo (3) neural network gaussian process (3) generative model (3) stochastic gradient descent (3) gradient descent (3) neural network training (3) batch size (2) recurrent neural network (2) energy-based model (2) evolution strategy (2) variance reduction (2) stochastic process (2) gradient estimation (2) mean field theory (2) batch normalization (2) diffusion model (2)

Papers

Scaling Exponents Across Parameterizations and Optimizers ICML 2024 Small-scale proxies for large-scale Transformer training instabilities ICLR 2024 Position: Levels of AGI for Operationalizing Progress on the Path to AGI ICML 2024 Variance-Reduced Gradient Estimation via Noise-Reuse in Online Evolution Strategies NIPS 2023 Reduce, Reuse, Recycle: Compositional Generation with Energy-Based Diffusion Models and MCMC ICML 2023 Wide Bayesian neural networks have a simple weight posterior: theory and accelerated sampling ICML 2022 Unbiased Gradient Estimation in Unrolled Computation Graphs with Persistent Evolution Strategies (Extended Abstract) IJCAI 2022 A Closer Look at Learned Optimization: Stability, Robustness, and Inductive Biases NIPS 2022 Fast Finite Width Neural Tangent Kernel ICML 2022 Score-Based Generative Modeling through Stochastic Differential Equations ICLR 2021 Unbiased Gradient Estimation in Unrolled Computation Graphs with Persistent Evolution Strategies ICML 2021 Whitening and Second Order Optimization Both Make Information in the Dataset Unusable During Training, and Can Reduce or Prevent Generalization ICML 2021 Reverse engineering learned optimizers reveals known and novel mechanisms NIPS 2021 Infinite attention: NNGP and NTK for deep attention networks ICML 2020 Finite Versus Infinite Neural Networks: an Empirical Study NIPS 2020 Neural Tangents: Fast and Easy Infinite Neural Networks in Python ICLR 2020 Your GAN is Secretly an Energy-based Model and You Should Use Discriminator Driven Latent Sampling NIPS 2020 Measuring the Effects of Data Parallelism on Neural Network Training JMLR 2019 Wide Neural Networks of Any Depth Evolve as Linear Models Under Gradient Descent NIPS 2019 Invertible Convolutional Flow NIPS 2019 A Mean Field Theory of Batch Normalization ICLR 2019 Bayesian Deep Convolutional Networks with Many Channels are Gaussian Processes ICLR 2019 Meta-Learning Update Rules for Unsupervised Representation Learning ICLR 2019 Adversarial Reprogramming of Neural Networks ICLR 2019 Guided evolutionary strategies: augmenting random search with surrogate gradients ICML 2019 Understanding and correcting pathologies in the training of learned optimizers ICML 2019 The Effect of Network Width on Stochastic Gradient Descent and Generalization: an Empirical Study ICML 2019 Deep Neural Networks as Gaussian Processes ICLR 2018 Sensitivity and Generalization in Neural Networks: an Empirical Study ICLR 2018 Generalizing Hamiltonian Monte Carlo with Neural Networks ICLR 2018 Dynamical Isometry and a Mean Field Theory of CNNs: How to Train 10,000-Layer Vanilla Convolutional Neural Networks ICML 2018 PCA of high dimensional random walks with comparison to neural network training NIPS 2018 Adversarial Examples that Fool both Computer Vision and Time-Limited Humans NIPS 2018 Input Switched Affine Networks: An RNN Architecture Designed for Interpretability ICML 2017 On the Expressive Power of Deep Neural Networks ICML 2017 Learned Optimizers that Scale and Generalize ICML 2017 REBAR: Low-variance, unbiased gradient estimates for discrete latent variable models NIPS 2017 SVCCA: Singular Vector Canonical Correlation Analysis for Deep Learning Dynamics and Interpretability NIPS 2017 Exponential expressivity in deep neural networks through transient chaos NIPS 2016 Deep Unsupervised Learning using Nonequilibrium Thermodynamics ICML 2015 Deep Knowledge Tracing NIPS 2015 Fast large-scale optimization by unifying stochastic gradient and quasi-Newton methods ICML 2014 Hamiltonian Monte Carlo Without Detailed Balance ICML 2014 Training sparse natural image models with a fast Gibbs sampler of an extended state space NIPS 2012