conftrace_

inference serving

5 papers

Explore in graph

Co-occurring keywords

throughput optimization (46) latency-accuracy tradeoff (2) dynamic adaptation (7) resource utilization (16) job completion time (12) gpu optimization (27) batch scheduling (4) interference-aware scheduling (1) model batching (1) distributed serving (1)

Papers

SuperServe: Fine-Grained Inference Serving for Unpredictable Workloads NSDI 2025
USHER: Holistic Interference Avoidance for Resource Optimized ML Inference OSDI 2024
MOSEL: Inference Serving Using Dynamic Modality Selection EMNLP 2024
dLoRA: Dynamically Orchestrating Requests and Adapters for LoRA LLM Serving OSDI 2024
Orca: A Distributed Serving System for Transformer-Based Generative Models OSDI 2022