Skip to content
conftrace_

iteration-level scheduling

1 paper

Explore in graph

Co-occurring keywords · papers mentioning both

distributed serving (1) transformer model (1) distributed serving system (1) selective batching (1) batch scheduling (1) inference serving (1) generative model (1) autoregressive generation (1)

Papers

Orca: A Distributed Serving System for Transformer-Based Generative Models OSDI 2022