conftrace_

memory optimization

114 papers

Explore in graph

Co-occurring keywords

model compression (3302) large language model (13587) memory efficiency (261) kv cache (74) inference optimization (219) inference efficiency (248) efficient inference (233) edge computing (210) deep neural network (1803) efficient computing (781)

Papers

LLMem: Estimating GPU Memory Usage for Fine-Tuning Pre-Trained LLMs IJCAI 2024
Beam-search SIEVE for low-memory speech recognition INTERSPEECH 2024
PyramidInfer: Pyramid KV Cache Compression for High-throughput LLM Inference ACL 2024
NACL: A General and Effective KV Cache Eviction Framework for LLM at Inference Time ACL 2024
ChunkAttention: Efficient Self-Attention with Prefix-Aware KV Cache and Two-Phase Partition ACL 2024
Layer-wise Importance Matters: Less Memory for Better Performance in Parameter-efficient Fine-tuning of Large Language Models EMNLP 2024
Fast and Memory-Efficient Video Diffusion Using Streamlined Inference NIPS 2024
Reducing Transformer Key-Value Cache Size with Cross-Layer Attention NIPS 2024
A Comprehensive Evaluation of Quantization Strategies for Large Language Models ACL 2024
Memorize Step by Step: Efficient Long-Context Prefilling with Incremental Memory and Decremental Chunk EMNLP 2024
Expanding Sparse Tuning for Low Memory Usage NIPS 2024
Fast Vector Query Processing for Large Datasets Beyond GPU Memory with Reordered Pipelining NSDI 2024
Pipeline Parallelism with Controllable Memory NIPS 2024
Towards Memory- and Time-Efficient Backpropagation for Training Spiking Neural Networks ICCV 2023
MCUFormer: Deploying Vision Tranformers on Microcontrollers with Limited Memory NIPS 2023
Accelerated On-Device Forward Neural Network Training with Module-Wise Descending Asynchronism NIPS 2023
Scissorhands: Exploiting the Persistence of Importance Hypothesis for LLM KV Cache Compression at Test Time NIPS 2023
Differentially Private Optimization on Large Model at Small Cost ICML 2023
Moccasin: Efficient Tensor Rematerialization for Neural Networks ICML 2023
UPSCALE: Unconstrained Channel Pruning ICML 2023
FlexGen: High-Throughput Generative Inference of Large Language Models with a Single GPU ICML 2023
Rockmate: an Efficient, Fast, Automatic and Generic Tool for Re-materialization in PyTorch ICML 2023
MODeL: Memory Optimizations for Deep Learning ICML 2023
Dynamic Stashing Quantization for Efficient Transformer Training EMNLP 2023
SMART: A High-Performance Adaptive Radix Tree for Disaggregated Memory OSDI 2023