James Lee-Thorp
7 papers · 2022–2024 · 4 conferences · across top CS/AI conferences
Achievements
Jump to papers ↓+3 more ↓ Show less ↑
π£ Hot Topic Early Bird π§ Keyword Pioneer π Conference Polyglot (4) π Interdisciplinary Bridge π Cross-Pollinator (3)
π
Renaissance Researcher
(5)
πΊοΈ
Taxonomy Completionist
(16)
π₯
Mega-Team
(45)
Conferences
EMNLP (3)
NAACL (2)
ICLR (1)
JMLR (1)
Top co-authors
Keywords
efficient computing
(3)
model compression
(2)
mixture of expert
(2)
neural network
(2)
conditional computation
(1)
efficient inference
(1)
distributed computing
(1)
fourier transform
(1)
sparse model
(1)
model scaling
(1)
feedforward network
(1)
parameter efficiency
(1)
data pipeline
(1)
inference speed
(1)
multi-query attention
(1)
inference speedup
(1)
transformer encoder
(1)
efficient transformer
(1)
long document
(1)
memory augmentation
(1)
Papers
Memory Augmented Language Models through Mixture of Word Experts
NAACL 2024
GQA: Training Generalized Multi-Query Transformer Models from Multi-Head Checkpoints
EMNLP 2023
CoLT5: Faster Long-Range Transformers with Conditional Computation
EMNLP 2023
Scaling Up Models and Data with t5x and seqio
JMLR 2023
Sparse Upcycling: Training Mixture-of-Experts from Dense Checkpoints
ICLR 2023
FNet: Mixing Tokens with Fourier Transforms
NAACL 2022
Sparse Mixers: Combining MoE and Mixing to build a more efficient BERT
EMNLP 2022