Mark Schmidt

Heavy-Tailed Class Imbalance and Why Adam Outperforms Gradient Descent on Language Models NeurIPS 2024 BiSLS/SPS: Auto-tune Step Sizes for Stable Bi-level Optimization NeurIPS 2023 Don't be so Monotone: Relaxing Stochastic Line Search in Over-Parameterized Models NeurIPS 2023 Noise Is Not the Main Factor Behind the Gap Between Sgd and Adam on Transformers, But Sign Descent Might Be ICLR 2023 Optimistic Thompson Sampling-based algorithms for episodic reinforcement learning UAI 2023 Searching for Optimal Per-Coordinate Step-sizes with Multidimensional Backtracking NeurIPS 2023 Simplifying Momentum-based Positive-definite Submanifold Optimization with Applications to Deep Learning ICML 2023 Target-based Surrogates for Stochastic Optimization ICML 2023 Homeomorphic-Invariance of EM: Non-Asymptotic Convergence in KL Divergence for Exponential Families via Mirror Descent (Extended Abstract) IJCAI 2022 Let's Make Block Coordinate Descent Converge Faster: Faster Greedy Rules, Message-Passing, Active-Set Complexity, and Superlinear Convergence JMLR 2022 AutoRetouch: Automatic Professional Face Retouching WACV 2021 Homeomorphic-Invariance of EM: Non-Asymptotic Convergence in KL Divergence for Exponential Families via Mirror Descent AISTATS 2021 Robust Asymmetric Learning in POMDPs ICML 2021 Tractable structured natural-gradient descent using local parameterizations ICML 2021 Fast and Furious Convergence: Stochastic Second Order Methods under Interpolation AISTATS 2020 Handling the Positive-Definite Constraint in the Bayesian Learning Rule ICML 2020 Regret Bounds without Lipschitz Continuity: Online Learning with Relative-Lipschitz Losses NeurIPS 2020 Are we there yet? Manifold identification of gradient-related proximal methods AISTATS 2019 Distributed Maximization of "Submodular plus Diversity" Functions for Multi-label Feature Selection on Huge Datasets AISTATS 2019 Fast and Faster Convergence of SGD for Over-Parameterized Models and an Accelerated Perceptron AISTATS 2019 Fast and Simple Natural-Gradient Variational Inference with Mixture of Exponential-family Approximations ICML 2019 Painless Stochastic Gradient: Interpolation, Line-Search, and Convergence Rates NeurIPS 2019 Online Learning Rate Adaptation with Hypergradient Descent ICLR 2018 SLANG: Fast Structured Covariance Approximations for Bayesian Deep Learning with Natural Gradient NeurIPS 2018 Where are the blobs: Counting by Localization with Point Supervision ECCV 2018 Horde of Bandits using Gaussian Markov Random Fields AISTATS 2017 Model-Independent Online Learning for Influence Maximization ICML 2017 Coordinate Descent Converges Faster with the Gauss-Southwell Rule Than Random Selection ICML 2015 Non-Uniform Stochastic Average Gradient Method for Training Conditional Random Fields AISTATS 2015 StopWasting My Gradients: Practical SVRG NeurIPS 2015 Block-Coordinate Frank-Wolfe Optimization for Structural SVMs ICML 2013 A Stochastic Gradient Method with an Exponential Convergence _Rate for Finite Training Sets NeurIPS 2012 On Sparse, Spectral and Other Parameterizations of Binary Probabilistic Models AISTATS 2012 Convergence Rates of Inexact Proximal-Gradient Methods for Convex Optimization NeurIPS 2011 Convex Structure Learning in Log-Linear Models: Beyond Pairwise Potentials AISTATS 2010 Modeling annotator expertise: Learning when everybody knows a bit of something AISTATS 2010 An interior-point stochastic approximation method and an L1-regularized delta rule NeurIPS 2008 Optimizing Costly Functions with Simple Constraints: A Limited-Memory Projected Quasi-Newton Algorithm AISTATS 2005