memory bandwidth

Accelerating LLM Inference Throughput via Asynchronous KV Cache Prefetching AAAI 2026 Break On Through to the Other Side: Pooling Memory Elastically with RamRyder OSDI 2026 Inference in the Shadows: Taming Memory Bandwidth Contention in Mobile LLM Inference with Sereno OSDI 2026 Svalinn: Overload Control in Large-Scale Servers with Multiple Resource Bottlenecks OSDI 2026 Disentangling the Dual Role of NIC Receive Rings OSDI 2025 Scaling Laws and Efficient Inference for Ternary Language Models ACL 2025 Break the Sequential Dependency of LLM Inference Using Lookahead Decoding ICML 2024 Faster Neighborhood Attention: Reducing the O(n^2) Cost of Self Attention at the Threadblock Level NeurIPS 2024 FiDO: Fusion-in-Decoder optimized for stronger performance and faster inference ACL 2023 ShRing: Networking with Shared Receive Rings OSDI 2023 Post training 4-bit quantization of convolutional networks for rapid-deployment NeurIPS 2019 Memory-Efficient Modeling and Search Techniques for Hardware ASR Decoders Interspeech 2016