Skip to content
conftrace_

inference serving

8 papers

Explore in graph

Co-occurring keywords · papers mentioning both

large language model (3) gpu memory management (2) request scheduling (2) throughput optimization (2) model serving (2) autoregressive generation (2) latency-accuracy tradeoff (1) resource efficiency (1) dynamic adaptation (1) resource utilization (1)

Papers

FastServe: Iteration-Level Preemptive Scheduling for Large Language Model Inference NSDI 2026 Libra: Flexible Request Partitioning and Scheduling for Serving Unbalanced and Dynamic LLM Workloads NSDI 2026 Prism: Cost-Efficient Multi-LLM Serving via GPU Memory Ballooning OSDI 2026 SuperServe: Fine-Grained Inference Serving for Unpredictable Workloads NSDI 2025 dLoRA: Dynamically Orchestrating Requests and Adapters for LoRA LLM Serving OSDI 2024 MOSEL: Inference Serving Using Dynamic Modality Selection EMNLP 2024 USHER: Holistic Interference Avoidance for Resource Optimized ML Inference OSDI 2024 Orca: A Distributed Serving System for Transformer-Based Generative Models OSDI 2022