latency reduction

Probe Pruning: Accelerating LLMs through Dynamic Pruning via Model-Probing ICLR 2025 Quaff: Quantized Parameter-Efficient Fine-Tuning under Outlier Spatial Stability Hypothesis ACL 2025 QUTE: Quantifying Uncertainty in TinyML models with Early-exit-assisted ensembles for model-monitoring ICML 2025 ReFrame: Layer Caching for Accelerated Inference in Real-Time Rendering ICML 2025 Scalable Offline ASR for Command-Style Dictation in Courtrooms Interspeech 2025 Self-Clocked Round-Robin Packet Scheduling NSDI 2025 Semantic Agreement Enables Efficient Open-Ended LLM Cascades EMNLP 2025 SeqPO-SiMT: Sequential Policy Optimization for Simultaneous Machine Translation ACL 2025 Speculative Decoding for Multi-Sample Inference EMNLP 2025 Approximate Caching for Efficiently Serving Text-to-Image Diffusion Models NSDI 2024 ConvoCache: Smart Re-Use of Chatbot Responses Interspeech 2024 Efficient LLM Scheduling by Learning to Rank NeurIPS 2024 Extremely efficient online query encoding for dense retrieval NAACL 2024 Hairpin: Rethinking Packet Loss Recovery in Edge-based Interactive Video Streaming NSDI 2024 MiLoRA: Efficient Mixture of Low-Rank Adaptation for Large Language Models Fine-tuning EMNLP 2024 MiniCache: KV Cache Compression in Depth Dimension for Large Language Models NeurIPS 2024 MOSEL: Inference Serving Using Dynamic Modality Selection EMNLP 2024 Online Speculative Decoding ICML 2024 Real-time Core-Periphery Guided ViT with Smart Data Layout Selection on Mobile Devices NeurIPS 2024 Self-Refining Diffusion Samplers: Enabling Parallelization via Parareal Iterations NeurIPS 2024 SIRIUS : Contexual Sparisty with Correction for Efficient LLMs NeurIPS 2024 wav2vec-S: Adapting Pre-trained Speech Models for Streaming ACL 2024 AlignAtt: Using Attention-based Audio-Translation Alignments as a Guide for Simultaneous Speech Translation Interspeech 2023 Attention as a Guide for Simultaneous Speech Translation ACL 2023 Enabling High Quality Real-Time Communications with Adaptive Frame-Rate NSDI 2023