Depen Morwani
9 papers · 2022–2025 · 4 venues · across top CS/AI venues
Authors are matched by name, so researchers who share a name ("Yang Liu", "Wei Wang") appear as a single profile and their paper counts, venues and achievements are combined. How identity is handled.
Achievements
Jump to papers ↓
🌍
Conference Polyglot · Bronze
🌉
Interdisciplinary Bridge · Bronze
🗺️
Taxonomy Completionist
(15)
❓
The Questioner
Venues
Top co-authors
Research topics
Machine Learning
(8)
Optimization & Theory · Machine Learning
(8)
Deep Learning
(7)
Architectures
(3)
Optimization & Theory · Deep Learning
(2)
Techniques
(2)
Artificial Intelligence
(1)
Mathematics & Optimization
(1)
Natural Language Processing
(1)
Core AI
(1)
Core Methods
(1)
Generation
(1)
Mathematics
(1)
Models
(1)
Optimization
(1)
Keywords
neural network
(2)
neural network optimization
(2)
language model
(2)
online learning
(1)
margin maximization
(1)
hessian approximation
(1)
gradient descent
(1)
learning rate
(1)
optimization algorithm
(1)
sparse solution
(1)
autoregressive model
(1)
ensemble method
(1)
adam optimizer
(1)
inductive bia
(1)
implicit bia
(1)
second-order optimization
(1)
scaling law
(1)
fourier feature
(1)
data parallelism
(1)
feature learning
(1)
Papers
A New Perspective on Shampoo's Preconditioner
ICLR 2025
Deconstructing What Makes a Good Optimizer for Autoregressive Language Models
ICLR 2025
How Does Critical Batch Size Scale in Pre-training?
ICLR 2025
SOAP: Improving and Stabilizing Shampoo using Adam for Language Modeling
ICLR 2025
Beyond Implicit Bias: The Insignificance of SGD Noise in Online Learning
ICML 2024
Feature emergence via margin maximization: case studies in algebraic tasks
ICLR 2024
Feature-Learning Networks Are Consistent Across Widths At Realistic Scales
NeurIPS 2023
Simplicity Bias in 1-Hidden Layer Neural Networks
NeurIPS 2023
Inductive Bias of Gradient Descent for Weight Normalized Smooth Homogeneous Neural Nets
ALT 2022