Joel Hestness
6 papers · 2017–2025 · 5 venues · across top CS/AI venues
Authors are matched by name, so researchers who share a name ("Yang Liu", "Wei Wang") appear as a single profile and their paper counts, venues and achievements are combined. How identity is handled.
Achievements
Jump to papers ↓
🌍
Conference Polyglot · Silver
🏃
Academic Marathon · Sprint
🌉
Interdisciplinary Bridge · Silver
🗺️
Taxonomy Completionist
(24)
🐣
Hot Topic Early Bird
Top co-authors
Research topics
Deep Learning
(6)
Machine Learning
(5)
Optimization & Theory · Deep Learning
(4)
Artificial Intelligence
(3)
Natural Language Processing
(3)
Application Areas
(3)
Architectures
(3)
Core AI
(2)
Optimization & Theory · Machine Learning
(2)
Techniques
(2)
Healthcare & Medicine
(1)
Speech & Audio
(1)
Applications
(1)
Clinical
(1)
Core Methods
(1)
Keywords
neural network optimization
(2)
neural network
(2)
model compression
(2)
domain adaptation
(1)
language modeling
(1)
hyperparameter transfer
(1)
speech recognition
(1)
sparsity level
(1)
efficient computing
(1)
keyword spotting
(1)
acoustic modeling
(1)
compositional generalization
(1)
training dynamics
(1)
instruction learning
(1)
sparse neural network
(1)
sparse training
(1)
deep learning
(1)
model scaling
(1)
learning rate schedule
(1)
machine translation
(1)
Papers
Straight to Zero: Why Linearly Decaying the Learning Rate to Zero Works Best for LLMs
ICLR 2025
MediSwift: Efficient Sparse Pre-trained Biomedical Language Models
ACL 2024
Normalization Layer Per-Example Gradients are Sufficient to Predict Gradient Noise Scale in Transformers
NeurIPS 2024
Sparse maximal update parameterization: A holistic approach to sparse training dynamics
NeurIPS 2024
Compositional Generalization for Primitive Substitutions
EMNLP 2019
Convolutional Recurrent Neural Networks for Small-Footprint Keyword Spotting
Interspeech 2017