Jascha Sohl-dickstein

Position: Levels of AGI for Operationalizing Progress on the Path to AGI ICML 2024 Scaling Exponents Across Parameterizations and Optimizers ICML 2024 Small-scale proxies for large-scale Transformer training instabilities ICLR 2024 Reduce, Reuse, Recycle: Compositional Generation with Energy-Based Diffusion Models and MCMC ICML 2023 Variance-Reduced Gradient Estimation via Noise-Reuse in Online Evolution Strategies NeurIPS 2023 A Closer Look at Learned Optimization: Stability, Robustness, and Inductive Biases NeurIPS 2022 Fast Finite Width Neural Tangent Kernel ICML 2022 Unbiased Gradient Estimation in Unrolled Computation Graphs with Persistent Evolution Strategies (Extended Abstract) IJCAI 2022 Wide Bayesian neural networks have a simple weight posterior: theory and accelerated sampling ICML 2022 Reverse engineering learned optimizers reveals known and novel mechanisms NeurIPS 2021 Score-Based Generative Modeling through Stochastic Differential Equations ICLR 2021 Unbiased Gradient Estimation in Unrolled Computation Graphs with Persistent Evolution Strategies ICML 2021 Whitening and Second Order Optimization Both Make Information in the Dataset Unusable During Training, and Can Reduce or Prevent Generalization ICML 2021 Finite Versus Infinite Neural Networks: an Empirical Study NeurIPS 2020 Infinite attention: NNGP and NTK for deep attention networks ICML 2020 Neural Tangents: Fast and Easy Infinite Neural Networks in Python ICLR 2020 Your GAN is Secretly an Energy-based Model and You Should Use Discriminator Driven Latent Sampling NeurIPS 2020 Adversarial Reprogramming of Neural Networks ICLR 2019 A Mean Field Theory of Batch Normalization ICLR 2019 Bayesian Deep Convolutional Networks with Many Channels are Gaussian Processes ICLR 2019 Guided evolutionary strategies: augmenting random search with surrogate gradients ICML 2019 Invertible Convolutional Flow NeurIPS 2019 Measuring the Effects of Data Parallelism on Neural Network Training JMLR 2019 Meta-Learning Update Rules for Unsupervised Representation Learning ICLR 2019 The Effect of Network Width on Stochastic Gradient Descent and Generalization: an Empirical Study ICML 2019 Understanding and correcting pathologies in the training of learned optimizers ICML 2019 Wide Neural Networks of Any Depth Evolve as Linear Models Under Gradient Descent NeurIPS 2019 Adversarial Examples that Fool both Computer Vision and Time-Limited Humans NeurIPS 2018 Deep Neural Networks as Gaussian Processes ICLR 2018 Dynamical Isometry and a Mean Field Theory of CNNs: How to Train 10,000-Layer Vanilla Convolutional Neural Networks ICML 2018 Generalizing Hamiltonian Monte Carlo with Neural Networks ICLR 2018 PCA of high dimensional random walks with comparison to neural network training NeurIPS 2018 Sensitivity and Generalization in Neural Networks: an Empirical Study ICLR 2018 Input Switched Affine Networks: An RNN Architecture Designed for Interpretability ICML 2017 Learned Optimizers that Scale and Generalize ICML 2017 On the Expressive Power of Deep Neural Networks ICML 2017 REBAR: Low-variance, unbiased gradient estimates for discrete latent variable models NeurIPS 2017 SVCCA: Singular Vector Canonical Correlation Analysis for Deep Learning Dynamics and Interpretability NeurIPS 2017 Exponential expressivity in deep neural networks through transient chaos NeurIPS 2016 Deep Knowledge Tracing NeurIPS 2015 Deep Unsupervised Learning using Nonequilibrium Thermodynamics ICML 2015 Fast large-scale optimization by unifying stochastic gradient and quasi-Newton methods ICML 2014 Hamiltonian Monte Carlo Without Detailed Balance ICML 2014 Training sparse natural image models with a fast Gibbs sampler of an extended state space NeurIPS 2012