Sumit Sanghai
9 papers · 2020–2024 · 6 venues · across top CS/AI venues
Authors are matched by name, so researchers who share a name ("Yang Liu", "Wei Wang") appear as a single profile and their paper counts, venues and achievements are combined. How identity is handled.
Achievements
Jump to papers ↓+1 more ↓ Show less ↑
🌍 Conference Polyglot · Silver 🌉 Interdisciplinary Bridge · Bronze 🗺️ Taxonomy Completionist (24) 🐣 Hot Topic Early Bird ❓ The Questioner
⚡
Prolific Year · Productive
Top co-authors
Research topics
Deep Learning
(7)
Machine Learning
(7)
Application Areas
(5)
Architectures
(5)
Artificial Intelligence
(4)
Core AI
(4)
Natural Language Processing
(3)
Optimization & Theory · Deep Learning
(3)
Core Methods
(2)
Generation
(2)
Models
(2)
Resources & Methods
(2)
Learning Types · Machine Learning
(1)
Learning Types · Deep Learning
(1)
Optimization & Theory · Machine Learning
(1)
Keywords
transformer architecture
(2)
transformer model
(2)
relative position encoding
(2)
language model
(2)
retrieval augmentation
(2)
language modeling
(1)
semantic parsing
(1)
machine translation
(1)
efficient inference
(1)
low-resource learning
(1)
efficient computing
(1)
vector quantization
(1)
memory efficiency
(1)
variational autoencoder
(1)
text generation
(1)
retrieval-augmented generation
(1)
inference optimization
(1)
conditional computation
(1)
feed-forward network
(1)
long-range transformers
(1)
Papers
Functional Interpolation for Relative Positions improves Long Context Transformers
ICLR 2024
MEMORY-VQ: Compression for Tractable Internet-Scale Memory
NAACL 2024
Arithmetic Sampling: Parallel Diverse Decoding for Large Language Models
ICML 2023
CoLT5: Faster Long-Range Transformers with Conditional Computation
EMNLP 2023
FiDO: Fusion-in-Decoder optimized for stronger performance and faster inference
ACL 2023
GQA: Training Generalized Multi-Query Transformer Models from Multi-Head Checkpoints
EMNLP 2023
Pre-computed memory or on-the-fly encoding? A hybrid approach to retrieval augmentation makes the most of your compute
ICML 2023
Generate-and-Retrieve: Use Your Predictions to Improve Retrieval for Semantic Parsing
COLING 2022
ETC: Encoding Long and Structured Inputs in Transformers
EMNLP 2020