Joshua Ainslie
25 papers · 2020–2026 · 8 venues · across top CS/AI venues
Authors are matched by name, so researchers who share a name ("Yang Liu", "Wei Wang") appear as a single profile and their paper counts, venues and achievements are combined. How identity is handled.
Achievements
Jump to papers ↓+9 more ↓ Show less ↑
πΊοΈ Taxonomy Completionist (49) π Academic Marathon Β· Sprint π Conference Polyglot Β· Silver π Interdisciplinary Bridge Β· Gold π£ Hot Topic Early Bird
π
Triple Crown
π₯
Mega-Team Β· Squad
π€
Dynamic Duo
(10)
π¬
Deep Specialist
(12)
π₯
Unstoppable Β· Bronze
π
Century Club Β· Bronze
β
The Questioner
ποΈ
Keyword Collector
(109)
β‘
Prolific Year Β· Productive
Top co-authors
Research topics
Deep Learning
(19)
Machine Learning
(15)
Architectures
(14)
Artificial Intelligence
(12)
Natural Language Processing
(11)
Core AI
(10)
Application Areas
(9)
Applications
(7)
Core Methods
(7)
Learning Types Β· Deep Learning
(7)
Generation
(5)
Optimization & Theory
(4)
Techniques
(4)
Learning Types Β· Machine Learning
(3)
Models
(3)
Keywords
question answering
(5)
model compression
(4)
transformer architecture
(4)
long sequence
(3)
efficient computing
(3)
transformer model
(3)
document understanding
(3)
text summarization
(3)
long document
(2)
information extraction
(2)
language model
(2)
compositional generalization
(2)
semantic parsing
(2)
vision transformer
(2)
retrieval augmentation
(2)
attention mechanism
(2)
multimodal learning
(2)
conditional computation
(2)
knowledge distillation
(2)
knowledge-intensive task
(2)
Papers
TIPSv2: Advancing Vision-Language Pretraining with Enhanced Patch-Text Alignment
CVPR 2026
Learning the RoPEs: Better 2D and 3D Position Encodings with STRING
ICML 2025
Linear Transformer Topological Masking with Graph Random Features
ICLR 2025
Functional Interpolation for Relative Positions improves Long Context Transformers
ICLR 2024
MEMORY-VQ: Compression for Tractable Internet-Scale Memory
NAACL 2024
A Suite of Generative Tasks for Multi-Level Multimodal Webpage Understanding
EMNLP 2023
CoLT5: Faster Long-Range Transformers with Conditional Computation
EMNLP 2023
Conditional Adapters: Parameter-efficient Transfer Learning with Fast Inference
NeurIPS 2023
FiDO: Fusion-in-Decoder optimized for stronger performance and faster inference
ACL 2023
FormNetV2: Multimodal Graph Contrastive Learning for Form Document Information Extraction
ACL 2023
GQA: Training Generalized Multi-Query Transformer Models from Multi-Head Checkpoints
EMNLP 2023
mLongT5: A Multilingual and Efficient Text-To-Text Transformer for Longer Sequences
EMNLP 2023
Pre-computed memory or on-the-fly encoding? A hybrid approach to retrieval augmentation makes the most of your compute
ICML 2023
Sparse Upcycling: Training Mixture-of-Experts from Dense Checkpoints
ICLR 2023
FNet: Mixing Tokens with Fourier Transforms
NAACL 2022
FormNet: Structural Encoding beyond Sequential Modeling in Form Document Information Extraction
ACL 2022
Generate-and-Retrieve: Use Your Predictions to Improve Retrieval for Semantic Parsing
COLING 2022
LongT5: Efficient Text-To-Text Transformer for Long Sequences
NAACL 2022
Making Transformers Solve Compositional Tasks
ACL 2022
Sparse Mixers: Combining MoE and Mixing to build a more efficient BERT
EMNLP 2022
Improving Compositional Generalization in Classification Tasks via Structure Annotations
ACL 2021
ReadTwice: Reading Very Large Documents with Memories
NAACL 2021
RealFormer: Transformer Likes Residual Attention
ACL 2021
Big Bird: Transformers for Longer Sequences
NeurIPS 2020
ETC: Encoding Long and Structured Inputs in Transformers
EMNLP 2020