inference acceleration

Graph-Structured Speculative Decoding ACL 2024 IceFormer: Accelerated Inference with Long-Sequence Transformers on CPUs ICLR 2024 Inevitable Trade-off between Watermark Strength and Speculative Sampling Efficiency for Language Models NeurIPS 2024 Investigating Acceleration of LLaMA Inference by Enabling Intermediate Layer Decoding via Instruction Tuning with ‘LITE’ NAACL 2024 Jointly-Learned Exit and Inference for a Dynamic Neural Network ICLR 2024 Kangaroo: Lossless Self-Speculative Decoding for Accelerating LLMs via Double Early Exiting NeurIPS 2024 LayerSkip: Enabling Early Exit Inference and Self-Speculative Decoding ACL 2024 Learning-to-Cache: Accelerating Diffusion Transformer via Layer Caching NeurIPS 2024 Lossless Acceleration of Large Language Model via Adaptive N-gram Parallel Decoding NAACL 2024 Make Some Noise: Unlocking Language Model Parallel Inference Capability through Noisy Training EMNLP 2024 MInference 1.0: Accelerating Pre-filling for Long-Context LLMs via Dynamic Sparse Attention NeurIPS 2024 Optimized Speculative Sampling for GPU Hardware Accelerators EMNLP 2024 Ouroboros: Generating Longer Drafts Phrase by Phrase for Faster Speculative Decoding EMNLP 2024 Plug-and-Play Diffusion Distillation CVPR 2024 Pruning Foundation Models for High Accuracy without Retraining EMNLP 2024 PyramidInfer: Pyramid KV Cache Compression for High-throughput LLM Inference ACL 2024 QEFT: Quantization for Efficient Fine-Tuning of LLMs EMNLP 2024 REST: Retrieval-Based Speculative Decoding NAACL 2024 Revisiting Token Pruning for Object Detection and Instance Segmentation WACV 2024 Search for Efficient Large Language Models NeurIPS 2024 Sequoia: Scalable and Robust Speculative Decoding NeurIPS 2024 ShiftAddLLM: Accelerating Pretrained LLMs via Post-Training Multiplication-Less Reparameterization NeurIPS 2024 SinSR: Diffusion-Based Image Super-Resolution in a Single Step CVPR 2024 SOI: Scaling Down Computational Complexity by Estimating Partial States of the Model NeurIPS 2024 SpecHub: Provable Acceleration to Multi-Draft Speculative Decoding EMNLP 2024