James Lee-Thorp
7 papers · 2022–2024 · 4 venues · across top CS/AI venues
Authors are matched by name, so researchers who share a name ("Yang Liu", "Wei Wang") appear as a single profile and their paper counts, venues and achievements are combined. How identity is handled.
Achievements
Jump to papers ↓
🌍
Conference Polyglot · Bronze
🌉
Interdisciplinary Bridge · Silver
🗺️
Taxonomy Completionist
(17)
🐣
Hot Topic Early Bird
👥
Mega-Team · Squad
Top co-authors
Research topics
Machine Learning
(7)
Application Areas
(6)
Deep Learning
(5)
Architectures
(5)
Artificial Intelligence
(4)
Core AI
(4)
Natural Language Processing
(3)
Optimization & Theory · Deep Learning
(3)
Optimization & Theory · Machine Learning
(2)
Resources & Methods
(2)
Computer Science
(1)
Applications
(1)
Generation
(1)
Learning Types
(1)
Keywords
efficient computing
(3)
neural network
(2)
model compression
(2)
mixture of expert
(2)
conditional computation
(1)
efficient inference
(1)
transformer architecture
(1)
fourier transform
(1)
long-range transformers
(1)
sparse model
(1)
dense checkpoint
(1)
feed-forward network
(1)
sparse training
(1)
model scaling
(1)
data pipeline
(1)
parameter efficiency
(1)
distributed computing
(1)
inference speed
(1)
multi-query attention
(1)
knowledge distillation
(1)
Papers
Memory Augmented Language Models through Mixture of Word Experts
NAACL 2024
CoLT5: Faster Long-Range Transformers with Conditional Computation
EMNLP 2023
GQA: Training Generalized Multi-Query Transformer Models from Multi-Head Checkpoints
EMNLP 2023
Scaling Up Models and Data with t5x and seqio
JMLR 2023
Sparse Upcycling: Training Mixture-of-Experts from Dense Checkpoints
ICLR 2023
FNet: Mixing Tokens with Fourier Transforms
NAACL 2022
Sparse Mixers: Combining MoE and Mixing to build a more efficient BERT
EMNLP 2022