Skip to content
conftrace_

request scheduling

11 papers

Explore in graph

Co-occurring keywords · papers mentioning both

large language model (5) tail latency (3) load balancing (3) latency optimization (2) inference serving (2) latency reduction (2) language model inference (1) head-of-line blocking (1) datacenter networking (1) throughput improvement (1)

Papers

Efficient LLM Serving on Commodity GPU Clusters with Data-Reduced Cross-Instance Orchestration OSDI 2026 JITServe: SLO-aware LLM Serving with Imprecise Request Information NSDI 2026 Libra: Flexible Request Partitioning and Scheduling for Serving Unbalanced and Dynamic LLM Workloads NSDI 2026 Simple Is Better: Multiplication May Be All You Need for LLM Request Scheduling OSDI 2026 Kamino: Efficient VM Allocation at Scale with Latency-Driven Cache-Aware Scheduling OSDI 2025 Semi-Clairvoyant Scheduling of Speculative Decoding Requests to Minimize LLM Inference Latency IJCAI 2025 SuperServe: Fine-Grained Inference Serving for Unpredictable Workloads NSDI 2025 Efficient LLM Scheduling by Learning to Rank NeurIPS 2024 RingLeader: Efficiently Offloading Intra-Server Orchestration to NICs NSDI 2023 Rearchitecting Linux Storage Stack for µs Latency and High Throughput OSDI 2021 Shinjuku: Preemptive Scheduling for μsecond-scale Tail Latency NSDI 2019