Joel Hestness
7 papers · 2017–2025 · 6 conferences · across top CS/AI conferences
Achievements
Jump to papers ↓+3 more ↓ Show less ↑
π Renaissance Researcher (5) π Interdisciplinary Bridge π§ Keyword Pioneer π Conference Polyglot (6) π Academic Marathon (8)
π£
Hot Topic Early Bird
π
Cross-Pollinator
(5)
πΊοΈ
Taxonomy Completionist
(28)
Conferences
NIPS (2)
ACL (1)
EMNLP (1)
ICLR (1)
IJCNLP (1)
INTERSPEECH (1)
Top co-authors
Keywords
machine translation
(2)
model compression
(2)
neural network
(2)
compositional generalization
(2)
few-shot learning
(2)
neural network optimization
(2)
attention map
(2)
primitive substitution
(2)
instruction learning
(2)
keyword spotting
(1)
efficient computing
(1)
attention mechanism
(1)
deep learning
(1)
acoustic modeling
(1)
hyperparameter transfer
(1)
natural language processing
(1)
sparse training
(1)
language modeling
(1)
sparse neural network
(1)
training dynamics
(1)
Papers
Straight to Zero: Why Linearly Decaying the Learning Rate to Zero Works Best for LLMs
ICLR 2025
Normalization Layer Per-Example Gradients are Sufficient to Predict Gradient Noise Scale in Transformers
NIPS 2024
MediSwift: Efficient Sparse Pre-trained Biomedical Language Models
ACL 2024
Sparse maximal update parameterization: A holistic approach to sparse training dynamics
NIPS 2024
Compositional Generalization for Primitive Substitutions
EMNLP 2019
Compositional Generalization for Primitive Substitutions
IJCNLP 2019
Convolutional Recurrent Neural Networks for Small-Footprint Keyword Spotting
INTERSPEECH 2017