inference efficiency

MiniCache: KV Cache Compression in Depth Dimension for Large Language Models NeurIPS 2024 MoE-I2: Compressing Mixture of Experts Models through Inter-Expert Pruning and Intra-Expert Low-Rank Decomposition EMNLP 2024 Multi-Task Inference: Can Large Language Models Follow Multiple Instructions at Once? ACL 2024 PikeLPN: Mitigating Overlooked Inefficiencies of Low-Precision Neural Networks CVPR 2024 PromptIntern: Saving Inference Costs by Internalizing Recurrent Prompt during Large Language Model Fine-tuning EMNLP 2024 QEFT: Quantization for Efficient Fine-Tuning of LLMs EMNLP 2024 QUIK: Towards End-to-end 4-Bit Inference on Generative Large Language Models EMNLP 2024 ReLU Strikes Back: Exploiting Activation Sparsity in Large Language Models ICLR 2024 Revisiting Convolution-free Transformer for Speech Recognition Interspeech 2024 Revisiting Non-Autoregressive Transformers for Efficient Image Synthesis CVPR 2024 Routing to the Expert: Efficient Reward-guided Ensemble of Large Language Models NAACL 2024 SIRIUS : Contexual Sparisty with Correction for Efficient LLMs NeurIPS 2024 Skeleton-of-Thought: Prompting LLMs for Efficient Parallel Generation ICLR 2024 Temperature-Centric Investigation of Speculative Decoding with Knowledge Distillation EMNLP 2024 TFMQ-DM: Temporal Feature Maintenance Quantization for Diffusion Models CVPR 2024 XC-Cache: Cross-Attending to Cached Context for Efficient LLM Inference EMNLP 2024 XMoE: Sparse Models with Fine-grained and Adaptive Expert Selection ACL 2024 You Only Cache Once: Decoder-Decoder Architectures for Language Models NeurIPS 2024 Accelerating Toeplitz Neural Network with Constant-time Inference Complexity EMNLP 2023 A Metric-Driven Approach to Conformer Layer Pruning for Efficient ASR Inference Interspeech 2023 AMOM: Adaptive Masking over Masking for Conditional Masked Language Model AAAI 2023 An Exploratory Study on Model Compression for Text-to-SQL ACL 2023 Are Straight-Through Gradients and Soft-Thresholding All You Need for Sparse Training? WACV 2023 Attend, Select and Eliminate: Accelerating Multi-turn Response Selection with Dual-attention-based Content Elimination ACL 2023 AutoGO: Automated Computation Graph Optimization for Neural Network Evolution NeurIPS 2023