Skip to content
conftrace_

Noam Shazeer

21 papers · 2015–2023 · 9 venues · across top CS/AI venues

Authors are matched by name, so researchers who share a name ("Yang Liu", "Wei Wang") appear as a single profile and their paper counts, venues and achievements are combined. How identity is handled.

Achievements

Jump to papers ↓
+12 more ↓ πŸ—ΊοΈ Taxonomy Completionist (35) πŸƒ Academic Marathon Β· Sprint 🌍 Conference Polyglot Β· Silver πŸŒ‰ Interdisciplinary Bridge Β· Silver 🐣 Hot Topic Early Bird
πŸ‘₯ Mega-Team Β· Legion 🌟 Keyword Trendsetter Combo (5) πŸ”¬ Deep Specialist (11) 🧭 Keyword Pioneer πŸ‘‘ Triple Crown πŸš€ Conference Pioneer πŸ”₯ Unstoppable Β· Bronze ❓ The Questioner (2) ⚑ Prolific Year Β· Productive πŸ’Ž Century Club Β· Bronze πŸ“ˆ Trend Setter πŸ—ƒοΈ Keyword Collector (77)

Venues

NeurIPS (5) JMLR (4) ICLR (3) ICML (3) EMNLP (2) ACL (1) CVPR (1) Interspeech (1) NAACL (1)

Top co-authors

Jakob Uszkoreit (6) Ashish Vaswani (6) Niki Parmar (6) Adam Roberts (5) LUKASZ KAISER (4) Sharan Narang (4) Colin Raffel (4) Ryan Sepassi (4) Hyung Won Chung (3) Noah Fiedel (3)

Research topics

Deep Learning (17) Architectures (13) Natural Language Processing (12) Machine Learning (11) Applications (7) Application Areas (6) Optimization & Theory Β· Machine Learning (6) Generation (5) Learning Types (5) Models (4) Techniques (4) Artificial Intelligence (3) Optimization & Theory Β· Deep Learning (3) Learning Paradigms (2) Resources & Methods (2)

Keywords

language model (4) transformer architecture (4) neural network (4) machine translation (3) model scaling (3) autoregressive model (3) neural machine translation (2) sequence transduction (2) parallel decoding (2) sequence model (2) large language model (2) recurrent neural network (2) distributed computing (2) mixture of expert (2) transformer model (2) image super-resolution (2) language modeling (1) curriculum learning (1) extractive summarization (1) question answering (1)

Papers

PaLM: Scaling Language Modeling with Pathways JMLR 2023 Scaling Up Models and Data with t5x and seqio JMLR 2023 Switch Transformers: Scaling to Trillion Parameter Models with Simple and Efficient Sparsity JMLR 2022 Do Transformer Modifications Transfer Across Implementations and Applications? EMNLP 2021 GShard: Scaling Giant Models with Conditional Computation and Automatic Sharding ICLR 2021 Searching for Efficient Transformers for Language Modeling NeurIPS 2021 Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer JMLR 2020 How Much Knowledge Can You Pack Into the Parameters of a Language Model? EMNLP 2020 Corpora Generation for Grammatical Error Correction NAACL 2019 Music Transformer: Generating Music with Long-Term Structure ICLR 2019 Adafactor: Adaptive Learning Rates with Sublinear Memory Cost ICML 2018 Blockwise Parallel Decoding for Deep Autoregressive Models NeurIPS 2018 Fast Decoding in Sequence Models Using Discrete Latent Variables ICML 2018 Generating Wikipedia by Summarizing Long Sequences ICLR 2018 HydraNets: Specialized Dynamic Architectures for Efficient Inference CVPR 2018 Image Transformer ICML 2018 Mesh-TensorFlow: Deep Learning for Supercomputers NeurIPS 2018 The Best of Both Worlds: Combining Recent Advances in Neural Machine Translation ACL 2018 Attention is All you Need NeurIPS 2017 NN-Grams: Unifying Neural Network and n-Gram Language Models for Speech Recognition Interspeech 2016 Scheduled Sampling for Sequence Prediction with Recurrent Neural Networks NeurIPS 2015