conftrace_

inference optimization

219 papers

Explore in graph

Co-occurring keywords

large language model (13587) model compression (3302) efficient computing (781) kv cache (74) speculative decoding (111) language model (4599) inference efficiency (248) inference acceleration (244) efficient inference (233) memory efficiency (261)

Papers

SOI: Scaling Down Computational Complexity by Estimating Partial States of the Model NIPS 2024
SGLang: Efficient Execution of Structured Language Model Programs NIPS 2024
Fast and Memory-Efficient Video Diffusion Using Streamlined Inference NIPS 2024
Are More LLM Calls All You Need? Towards the Scaling Properties of Compound AI Systems NIPS 2024
Deep Dependency Networks and Advanced Inference Schemes for Multi-Label Classification AISTATS 2024
NACL: A General and Effective KV Cache Eviction Framework for LLM at Inference Time ACL 2024
Faster Vocoder: a multi threading approach to achieve low latency during TTS Inference INTERSPEECH 2024
Adaptive Deep Neural Network Inference Optimization With EENet WACV 2024
SLiM: Speculative Decoding with Hypothesis Reduction NAACL 2024
Predicting Rewards Alongside Tokens: Non-disruptive Parameter Insertion for Efficient Inference Intervention in Large Language Model EMNLP 2024
Overload: Latency Attacks on Object Detection for Edge Devices CVPR 2024
RLtools: A Fast, Portable Deep Reinforcement Learning Library for Continuous Control JMLR 2024
Fortify the Shortest Stave in Attention: Enhancing Context Awareness of Large Language Models for Effective Tool Use ACL 2024
ChunkAttention: Efficient Self-Attention with Prefix-Aware KV Cache and Two-Phase Partition ACL 2024
LayerSkip: Enabling Early Exit Inference and Self-Speculative Decoding ACL 2024
Unlocking Efficiency in Large Language Model Inference: A Comprehensive Survey of Speculative Decoding ACL 2024
QUIK: Towards End-to-end 4-Bit Inference on Generative Large Language Models EMNLP 2024
Optimized Speculative Sampling for GPU Hardware Accelerators EMNLP 2024
EAGLE-2: Faster Inference of Language Models with Dynamic Draft Trees EMNLP 2024
VPTQ: Extreme Low-bit Vector Post-Training Quantization for Large Language Models EMNLP 2024
Towards Fast Multilingual LLM Inference: Speculative Decoding and Specialized Drafters EMNLP 2024
KVQuant: Towards 10 Million Context Length LLM Inference with KV Cache Quantization NIPS 2024
Efficient LLM Scheduling by Learning to Rank NIPS 2024
Cascade Speculative Drafting for Even Faster LLM Inference NIPS 2024
FlattenQuant: Breaking through the Inference Compute-bound for Large Language Models with Per-tensor Quantization COLING 2024