conftrace_

sparse layer

3 papers

Explore in graph

Co-occurring keywords · papers mentioning both

balanced assignment (1) weight hashing (1) token routing (1) transformer model (1) large language model (1) feed-forward layer (1) expert routing (1) sparse attention (1) mixture of expert (1) sparse model (1)

Papers

BASE Layers: Simplifying Training of Large, Sparse Models ICML 2021 Hash Layers For Large Sparse Models NeurIPS 2021 Sparse is Enough in Scaling Transformers NeurIPS 2021