conftrace_

distributed serving

1 paper

Explore in graph

Co-occurring keywords · papers mentioning both

iteration-level scheduling (1) transformer model (1) distributed serving system (1) selective batching (1) batch scheduling (1) inference serving (1) generative model (1) autoregressive generation (1)

Papers

Orca: A Distributed Serving System for Transformer-Based Generative Models OSDI 2022