conftrace_

model serving

21 papers

Explore in graph

Co-occurring keywords

large language model (13587) latency optimization (119) deep neural network (1803) throughput optimization (46) inference optimization (219) tail latency (25) knowledge distillation (3725) model compression (3302) dynamic scheduling (7) dynamic programming (274)

Papers

Breaking the Resource Monopoly: LLM Post-Training and Serving with Modest Data and Compute AAAI 2026
Deep Kernel Fusion for Transformers ACL 2026
RecStream: Graph-aware Stream Management for Concurrent Recommendation Model Online Serving COLING 2025
BlitzScale: Fast and Live Large Model Autoscaling with O(1) Host Caching OSDI 2025
GPU-Disaggregated Serving for Deep Learning Recommendation Models at Scale NSDI 2025
Dynamic Operator Optimization for Efficient Multi-Tenant LoRA Model Serving AAAI 2025
Breaking the Resource Monopoly from Industries: Sustainable and Reliable LLM Serving by Recycling Outdated and Resource-Constrained GPUs AAAI 2025
In Defense of Structural Sparse Adapters for Concurrent LLM Serving EMNLP 2024
Efficient Multi-task LLM Quantization and Serving for Multiple LoRA Adapters NIPS 2024
ScaleLLM: A Resource-Frugal LLM Serving Framework by Optimizing End-to-End Efficiency EMNLP 2024
Taming Throughput-Latency Tradeoff in LLM Inference with Sarathi-Serve OSDI 2024
Complementary Knowledge Distillation for Robust and Privacy-Preserving Model Serving in Vertical Federated Learning AAAI 2024
RelayAttention for Efficient Large Language Model Serving with Long System Prompts ACL 2024
Xinference: Making Large Model Serving Easy EMNLP 2024
Llumnix: Dynamic Scheduling for Large Language Model Serving OSDI 2024
MOSEL: Inference Serving Using Dynamic Modality Selection EMNLP 2024
Sequoia: Scalable and Robust Speculative Decoding NIPS 2024
SHEPHERD: Serving DNNs in the Wild NSDI 2023
Cocktail: A Multidimensional Optimization for Model Serving in Cloud NSDI 2022
Serving DNNs like Clockwork: Performance Predictability from the Bottom Up OSDI 2020
A Tensor Compiler for Unified Machine Learning Prediction Serving OSDI 2020