Atli Kosson
6 papers · 2019–2024 · 3 conferences · across top CS/AI conferences
Achievements
Jump to papers ↓+1 more ↓ Show less ↑
π Interdisciplinary Bridge π§ Keyword Pioneer π Conference Polyglot (3) π Academic Marathon (5) π Cross-Pollinator (12)
πΊοΈ
Taxonomy Completionist
(13)
Conferences
NIPS (4)
AAAI (1)
ICML (1)
Top co-authors
Keywords
neural network optimization
(2)
neural network training
(2)
image classification
(1)
batch normalization
(1)
language modeling
(1)
efficient computing
(1)
automatic differentiation
(1)
gradient descent
(1)
learning rate
(1)
adam optimizer
(1)
model training
(1)
scaling law
(1)
stochastic weight averaging
(1)
learning rate warmup
(1)
gpt training
(1)
model scaling
(1)
compute efficiency
(1)
learning rate schedule
(1)
training schedule
(1)
hidden activation
(1)
Papers
Analyzing & Reducing the Need for Learning Rate Warmup in GPT Training
NIPS 2024
Scaling Laws and Compute-Optimal Training Beyond Fixed Training Durations
NIPS 2024
Ghost Noise for Regularizing Deep Neural Networks
AAAI 2024
Rotational Equilibrium: How Weight Decay Balances Learning Across Neural Networks
ICML 2024
Multiplication-Free Transformer Training via Piecewise Affine Operations
NIPS 2023
Online Normalization for Training Neural Networks
NIPS 2019