Atli Kosson
6 papers · 2019–2024 · 3 venues · across top CS/AI venues
Authors are matched by name, so researchers who share a name ("Yang Liu", "Wei Wang") appear as a single profile and their paper counts, venues and achievements are combined. How identity is handled.
Achievements
Jump to papers ↓
🌍
Conference Polyglot · Bronze
🏃
Academic Marathon · Sprint
🗺️
Taxonomy Completionist
(14)
Venues
Top co-authors
Research topics
Keywords
neural network optimization
(2)
neural network training
(2)
image classification
(1)
batch normalization
(1)
language modeling
(1)
efficient computing
(1)
automatic differentiation
(1)
gradient descent
(1)
learning rate
(1)
adam optimizer
(1)
model training
(1)
scaling law
(1)
weight decay
(1)
stochastic weight averaging
(1)
learning rate warmup
(1)
gpt training
(1)
model scaling
(1)
compute efficiency
(1)
learning rate schedule
(1)
training schedule
(1)
Papers
Analyzing & Reducing the Need for Learning Rate Warmup in GPT Training
NeurIPS 2024
Ghost Noise for Regularizing Deep Neural Networks
AAAI 2024
Rotational Equilibrium: How Weight Decay Balances Learning Across Neural Networks
ICML 2024
Scaling Laws and Compute-Optimal Training Beyond Fixed Training Durations
NeurIPS 2024
Multiplication-Free Transformer Training via Piecewise Affine Operations
NeurIPS 2023
Online Normalization for Training Neural Networks
NeurIPS 2019