Oscillation-Reduced MXFP4 Training for Vision Transformers

Yuxiang Chen; Haocheng Xi; Jun Zhu; Jianfei Chen

2025 ICML ICML 2025

Oscillation-Reduced MXFP4 Training for Vision Transformers

Abstract

No abstract available. No abstract was captured for this paper when its proceedings were scraped - most often an older edition published without one online. How the corpus is built.

Authors

Yuxiang Chen, Haocheng Xi, Jun Zhu, Jianfei Chen

Topics

Machine Learning > Optimization & Theory > Optimization Machine Learning > Application Areas > Efficient Computing Deep Learning > Architectures > Transformers

Keywords

vision transformer weight oscillation adaptive ramping optimizer (q-ramping) ema quantizer (q-ema) fp4 precision microscaling (mx) data format

Download PDF