conftrace_

inference optimization

219 papers

Explore in graph

Co-occurring keywords

large language model (13587) model compression (3302) efficient computing (781) kv cache (74) speculative decoding (111) language model (4599) inference efficiency (248) inference acceleration (244) efficient inference (233) memory efficiency (261)

Papers

Selective Perception: Learning Concise State Descriptions for Language Model Actors NAACL 2024
LLMRefine: Pinpointing and Refining Large Language Models via Fine-Grained Actionable Feedback NAACL 2024
DynaThink: Fast or Slow? A Dynamic Decision-Making Framework for Large Language Models EMNLP 2024
CHESS: Optimizing LLM Inference via Channel-Wise Thresholding and Selective Sparsification EMNLP 2024
SpecHub: Provable Acceleration to Multi-Draft Speculative Decoding EMNLP 2024
Sprout: Green Generative AI with Carbon-Efficient LLM Inference EMNLP 2024
TensorOpera Router: A Multi-Model Router for Efficient LLM Inference EMNLP 2024
Query-OPT: Optimizing Inference of Large Language Models via Multi-Query Instructions in Meeting Summarization EMNLP 2024
In-Context Former: Lightning-fast Compressing Context for Large Language Model EMNLP 2024
Temperature-Centric Investigation of Speculative Decoding with Knowledge Distillation EMNLP 2024
Xinference: Making Large Model Serving Easy EMNLP 2024
KV Cache is 1 Bit Per Channel: Efficient Large Language Model Inference with Coupled Quantization NIPS 2024
Toward Efficient Inference for Mixture of Experts NIPS 2024
Retaining Key Information under High Compression Ratios: Query-Guided Compressor for LLMs ACL 2024
PyramidInfer: Pyramid KV Cache Compression for High-throughput LLM Inference ACL 2024
Revisiting Non-Autoregressive Transformers for Efficient Image Synthesis CVPR 2024
LOOK-M: Look-Once Optimization in KV Cache for Efficient Multimodal Long-Context Inference EMNLP 2024
When Compression Meets Model Compression: Memory-Efficient Double Compression for Large Language Models EMNLP 2024
A Thorough Examination of Decoding Methods in the Era of LLMs EMNLP 2024
MOSEL: Inference Serving Using Dynamic Modality Selection EMNLP 2024
Normalized Narrow Jump To Conclusions: Normalized Narrow Shortcuts for Parameter Efficient Early Exit Transformer Prediction EMNLP 2024
Sing it, Narrate it: Quality Musical Lyrics Translation EMNLP 2024
Fast Matrix Multiplications for Lookup Table-Quantized LLMs EMNLP 2024
Eigen Attention: Attention in Low-Rank Space for KV Cache Compression EMNLP 2024
MInference 1.0: Accelerating Pre-filling for Long-Context LLMs via Dynamic Sparse Attention NIPS 2024