Skip to content
conftrace_
2024 ICML ICML 2024

Revisiting Zeroth-Order Optimization for Memory-Efficient LLM Fine-Tuning: A Benchmark

Abstract

No abstract available. No abstract was captured for this paper when its proceedings were scraped - most often an older edition published without one online. How the corpus is built.

🐣 Hot Topic Early Bird - zeroth-order optimization

Authors

Yihua Zhang, Pingzhi Li, Junyuan Hong, Jiaxiang Li, Yimeng Zhang, Wenqing Zheng, Pin-Yu Chen, Jason D. Lee, Wotao Yin, Mingyi Hong, Zhangyang Wang, Sijia Liu, Tianlong Chen

Topics

Machine Learning > Optimization & Theory > Optimization Machine Learning > Application Areas > Efficient Computing Artificial Intelligence > Core AI > Large Language Models

Keywords

zeroth-order optimization hybrid training large language model gradient sparsity memory-efficient fine-tuning block-wise descent
Download PDF