Frederik Kunstner
8 papers · 2018–2024 · 4 conferences · across top CS/AI conferences
Achievements
Jump to papers ↓+2 more ↓ Show less ↑
🌉 Interdisciplinary Bridge 🏃 Academic Marathon (6) 🐝 Cross-Pollinator (15) 🌍 Conference Polyglot (4) 🧭 Keyword Pioneer
🐣
Hot Topic Early Bird
🔥
Unstoppable
(7)
Conferences
NIPS (4)
ICLR (2)
AISTATS (1)
IJCAI (1)
Top co-authors
Keywords
expectation maximization
(2)
mirror descent
(2)
natural gradient
(2)
kl divergence
(2)
class imbalance
(1)
fisher information
(1)
gradient descent
(1)
exponential family
(1)
natural gradient descent
(1)
exponential families
(1)
fisher information matrix
(1)
hyperparameter tuning
(1)
second-order optimization
(1)
latent variable model
(1)
convergence rate
(1)
adam optimizer
(1)
heavy-tailed distribution
(1)
language model
(1)
second-order method
(1)
uncertainty estimation
(1)
Papers
Heavy-Tailed Class Imbalance and Why Adam Outperforms Gradient Descent on Language Models
NIPS 2024
Noise Is Not the Main Factor Behind the Gap Between Sgd and Adam on Transformers, But Sign Descent Might Be
ICLR 2023
Searching for Optimal Per-Coordinate Step-sizes with Multidimensional Backtracking
NIPS 2023
Homeomorphic-Invariance of EM: Non-Asymptotic Convergence in KL Divergence for Exponential Families via Mirror Descent (Extended Abstract)
IJCAI 2022
Homeomorphic-Invariance of EM: Non-Asymptotic Convergence in KL Divergence for Exponential Families via Mirror Descent
AISTATS 2021
BackPACK: Packing more into Backprop
ICLR 2020
Limitations of the empirical Fisher approximation for natural gradient descent
NIPS 2019
SLANG: Fast Structured Covariance Approximations for Bayesian Deep Learning with Natural Gradient
NIPS 2018