conftrace_

selective batching

1 paper

Explore in graph

Co-occurring keywords · papers mentioning both

distributed serving system (1) transformer model (1) distributed serving (1) iteration-level scheduling (1) batch scheduling (1) inference serving (1) generative model (1) autoregressive generation (1)

Papers

Orca: A Distributed Serving System for Transformer-Based Generative Models OSDI 2022