conftrace_

forward-mode differentiation

4 papers

Explore in graph

Also known as

FMD

Co-occurring keywords

hyperparameter optimization (406) language model finetuning (14) parameter-efficient finetuning (59) model finetuning (26) loss surface (22) large language model (13587) gpu parallelism (2) forward-mode automatic differentiation (1) constant memory (5) reverse-mode differentiation (2)

Papers

Second-order forward-mode optimization of recurrent neural networks for neuroscience NIPS 2024
Thinking Forward: Memory-Efficient Federated Finetuning of Language Models NIPS 2024
Gradient-based Hyperparameter Optimization Over Long Horizons NIPS 2021
Forward and Reverse Gradient-Based Hyperparameter Optimization ICML 2017