conftrace_

distributed llm serving

2 papers

Explore in graph

Co-occurring keywords · papers mentioning both

consumer-grade gpu (1) prefill-decode scheduling (1) transmission scheduling (1) inference latency optimization (1) chunking algorithm (1) kv cache reuse (1) prompt scheduling (1) load balancing (1) inference optimization (1)

Papers

Distributed LLM Serving on Consumer-Grade GPUs by Reconciling Computation and Communication EMNLP 2025 Preble: Efficient Distributed Prompt Scheduling for LLM Serving ICLR 2025