Frederik Kunstner
8 papers · 2018–2024 · 4 venues · across top CS/AI venues
Authors are matched by name, so researchers who share a name ("Yang Liu", "Wei Wang") appear as a single profile and their paper counts, venues and achievements are combined. How identity is handled.
Achievements
Jump to papers ↓+1 more ↓ Show less ↑
🌍 Conference Polyglot · Bronze 🏃 Academic Marathon · Sprint 🌉 Interdisciplinary Bridge · Bronze 🗺️ Taxonomy Completionist (14) 🐣 Hot Topic Early Bird
🔥
Unstoppable · Bronze
Venues
Top co-authors
Research topics
Keywords
natural gradient
(2)
expectation maximization
(2)
adam optimizer
(2)
kl divergence
(2)
mirror descent
(2)
automatic differentiation
(1)
class imbalance
(1)
fisher information
(1)
gradient descent
(1)
exponential family
(1)
natural gradient descent
(1)
exponential families
(1)
fisher information matrix
(1)
hyperparameter tuning
(1)
second-order optimization
(1)
step size optimization
(1)
latent variable model
(1)
convergence rate
(1)
heavy-tailed distribution
(1)
language model
(1)
Papers
Heavy-Tailed Class Imbalance and Why Adam Outperforms Gradient Descent on Language Models
NeurIPS 2024
Noise Is Not the Main Factor Behind the Gap Between Sgd and Adam on Transformers, But Sign Descent Might Be
ICLR 2023
Searching for Optimal Per-Coordinate Step-sizes with Multidimensional Backtracking
NeurIPS 2023
Homeomorphic-Invariance of EM: Non-Asymptotic Convergence in KL Divergence for Exponential Families via Mirror Descent (Extended Abstract)
IJCAI 2022
Homeomorphic-Invariance of EM: Non-Asymptotic Convergence in KL Divergence for Exponential Families via Mirror Descent
AISTATS 2021
BackPACK: Packing more into Backprop
ICLR 2020
Limitations of the empirical Fisher approximation for natural gradient descent
NeurIPS 2019
SLANG: Fast Structured Covariance Approximations for Bayesian Deep Learning with Natural Gradient
NeurIPS 2018