Kaifeng Lyu
22 papers · 2019–2025 · 3 venues · across top CS/AI venues
Authors are matched by name, so researchers who share a name ("Yang Liu", "Wei Wang") appear as a single profile and their paper counts, venues and achievements are combined. How identity is handled.
Achievements
Jump to papers ↓+7 more ↓ Show less ↑
πΊοΈ Taxonomy Completionist (33) π Academic Marathon Β· Sprint π Conference Polyglot Β· Bronze π Interdisciplinary Bridge Β· Bronze π£ Hot Topic Early Bird
π€
Dynamic Duo
(10)
π¬
Deep Specialist
(10)
β‘
Prolific Year Β· Productive
π₯
Unstoppable Β· Bronze
β
The Questioner
π
Century Club Β· Bronze
ποΈ
Keyword Collector
(96)
Venues
Top co-authors
Research topics
Machine Learning
(18)
Optimization & Theory Β· Machine Learning
(16)
Deep Learning
(10)
Optimization & Theory Β· Deep Learning
(6)
Artificial Intelligence
(4)
Application Areas
(3)
Architectures
(3)
Core AI
(3)
Mathematics & Optimization
(2)
Core Methods
(2)
Learning Types Β· Deep Learning
(2)
Learning Types Β· Machine Learning
(2)
Optimization
(2)
Techniques
(2)
Computer Vision
(1)
Keywords
gradient descent
(7)
neural network optimization
(6)
learning rate
(5)
stochastic differential equation
(3)
batch normalization
(3)
margin maximization
(2)
implicit bia
(2)
stochastic gradient descent
(2)
weight decay
(2)
safety alignment
(2)
matrix factorization
(1)
feature attribution
(1)
incremental learning
(1)
model interpretability
(1)
transfer learning
(1)
feature learning
(1)
distribution shift
(1)
knowledge distillation
(1)
binary classification
(1)
model robustness
(1)
Papers
A Multi-Power Law for Loss Curve Prediction Across Learning Rate Schedules
ICLR 2025
Efficient stagewise pretraining via progressive subnetworks
ICLR 2025
Feature Averaging: An Implicit Bias of Gradient Descent Leading to Non-Robustness in Neural Networks
ICLR 2025
RNNs are not Transformers (Yet): The Key Bottleneck on In-Context Retrieval
ICLR 2025
Safety Alignment Should be Made More Than Just a Few Tokens Deep
ICLR 2025
Towards Understanding Text Hallucination of Diffusion Models via Local Generation Bias
ICLR 2025
Weak-to-Strong Generalization Even in Random Feature Networks, Provably
ICML 2025
A Quadratic Synchronization Rule for Distributed Deep Learning
ICLR 2024
Dichotomy of Early and Late Phase Implicit Biases Can Provably Induce Grokking
ICLR 2024
DistillSpec: Improving Speculative Decoding via Knowledge Distillation
ICLR 2024
Keeping LLMs Aligned After Fine-tuning: The Crucial Role of Prompt Templates
NeurIPS 2024
The Marginal Value of Momentum for Small Learning Rate SGD
ICLR 2024
Understanding Incremental Learning of Gradient Descent: A Fine-grained Analysis of Matrix Sensing
ICML 2023
Why (and When) does Local SGD Generalize Better than SGD?
ICLR 2023
New Definitions and Evaluations for Saliency Methods: Staying Intrinsic, Complete and Sound
NeurIPS 2022
On the SDEs and Scaling Rules for Adaptive Gradient Algorithms
NeurIPS 2022
Understanding the Generalization Benefit of Normalization Layers: Sharpness Reduction
NeurIPS 2022
Gradient Descent on Two-layer Nets: Margin Maximization and Simplicity Bias
NeurIPS 2021
Towards Resolving the Implicit Bias of Gradient Descent for Matrix Factorization: Greedy Low-Rank Learning
ICLR 2021
Gradient Descent Maximizes the Margin of Homogeneous Neural Networks
ICLR 2020
Reconciling Modern Deep Learning with Traditional Optimization Analyses: The Intrinsic Learning Rate
NeurIPS 2020
Theoretical Analysis of Auto Rate-Tuning by Batch Normalization
ICLR 2019