model serving

Breaking the Resource Monopoly: LLM Post-Training and Serving with Modest Data and Compute AAAI 2026 Deep Kernel Fusion for Transformers ACL 2026 FlexLLM: Token-Level Co-Serving of LLM Inference and Finetuning with SLO Guarantees NSDI 2026 Prism: Cost-Efficient Multi-LLM Serving via GPU Memory Ballooning OSDI 2026 BlitzScale: Fast and Live Large Model Autoscaling with O(1) Host Caching OSDI 2025 Breaking the Resource Monopoly from Industries: Sustainable and Reliable LLM Serving by Recycling Outdated and Resource-Constrained GPUs AAAI 2025 Dynamic Operator Optimization for Efficient Multi-Tenant LoRA Model Serving AAAI 2025 GPU-Disaggregated Serving for Deep Learning Recommendation Models at Scale NSDI 2025 HexGen-2: Disaggregated Generative Inference of LLMs in Heterogeneous Environment ICLR 2025 RecStream: Graph-aware Stream Management for Concurrent Recommendation Model Online Serving COLING 2025 Round Attention: A Novel Round-Level Attention Mechanism to Accelerate LLM Inference ACML 2025 Complementary Knowledge Distillation for Robust and Privacy-Preserving Model Serving in Vertical Federated Learning AAAI 2024 Efficient Multi-task LLM Quantization and Serving for Multiple LoRA Adapters NeurIPS 2024 In Defense of Structural Sparse Adapters for Concurrent LLM Serving EMNLP 2024 Llumnix: Dynamic Scheduling for Large Language Model Serving OSDI 2024 MOSEL: Inference Serving Using Dynamic Modality Selection EMNLP 2024 RelayAttention for Efficient Large Language Model Serving with Long System Prompts ACL 2024 ScaleLLM: A Resource-Frugal LLM Serving Framework by Optimizing End-to-End Efficiency EMNLP 2024 Sequoia: Scalable and Robust Speculative Decoding NeurIPS 2024 Taming Throughput-Latency Tradeoff in LLM Inference with Sarathi-Serve OSDI 2024 Xinference: Making Large Model Serving Easy EMNLP 2024 SHEPHERD: Serving DNNs in the Wild NSDI 2023 Cocktail: A Multidimensional Optimization for Model Serving in Cloud NSDI 2022 A Tensor Compiler for Unified Machine Learning Prediction Serving OSDI 2020 Serving DNNs like Clockwork: Performance Predictability from the Bottom Up OSDI 2020