Kazuki Irie
23 papers · 2016–2025 · 6 venues · across top CS/AI venues
Authors are matched by name, so researchers who share a name ("Yang Liu", "Wei Wang") appear as a single profile and their paper counts, venues and achievements are combined. How identity is handled.
Achievements
Jump to papers ↓+11 more ↓ Show less ↑
🗺️ Taxonomy Completionist (41) 🏃 Academic Marathon · Sprint 🌍 Conference Polyglot · Silver 🌉 Interdisciplinary Bridge · Silver 🐣 Hot Topic Early Bird
🤝
Dynamic Duo
(13)
🧬
Topic Evolution
🏆
Keyword Champion
(2)
👑
Triple Crown
🚀
Conference Pioneer
🔥
Unstoppable · Bronze
❓
The Questioner
⚡
Prolific Year · Productive
💎
Century Club · Bronze
🌈
Renaissance Researcher
(7)
🗃️
Keyword Collector
(105)
Top co-authors
Research topics
Deep Learning
(15)
Machine Learning
(15)
Architectures
(13)
Optimization & Theory · Machine Learning
(9)
Core Methods
(7)
Artificial Intelligence
(6)
Natural Language Processing
(6)
Core AI
(6)
Speech & Audio
(4)
Generation
(4)
Optimization & Theory · Deep Learning
(4)
Recognition
(4)
Models
(3)
Techniques
(3)
Computer Vision
(2)
Keywords
attention mechanism
(7)
language model
(6)
fast weight programmer
(4)
language modeling
(4)
recurrent neural network
(4)
systematic generalization
(3)
speech recognition
(3)
long short-term memory
(3)
mixture of expert
(3)
automatic speech recognition
(3)
reinforcement learning
(2)
universal transformer
(2)
linear transformer
(2)
positional encoding
(2)
bidirectional lstm
(2)
neural network
(2)
bayesian learning
(1)
online learning
(1)
self-attention mechanism
(1)
contrastive learning
(1)
Papers
Why Are Positional Encodings Nonessential for Deep Autoregressive Transformers? A Petroglyph Revisited
ACL 2025
Dissecting the Interplay of Attention Paths in a Statistical Mechanics Theory of Transformers
NeurIPS 2024
Exploring the Promise and Limits of Real-Time Recurrent Learning
ICLR 2024
MoEUT: Mixture-of-Experts Universal Transformers
NeurIPS 2024
SwitchHead: Accelerating Transformers with Mixture-of-Experts Attention
NeurIPS 2024
Approximating Two-Layer Feedforward Networks for Efficient Transformers
EMNLP 2023
Contrastive Training of Complex-Valued Autoencoders for Object Discovery
NeurIPS 2023
Images as Weight Matrices: Sequential Image Generation Through Synaptic Learning Rules
ICLR 2023
Practical Computational Power of Linear Transformers and Their Recurrent and Self-Referential Extensions
EMNLP 2023
A Modern Self-Referential Weight Matrix That Learns to Modify Itself
ICML 2022
CTL++: Evaluating Generalization on Never-Seen Compositional Patterns of Known Functions, and Compatibility of Neural Representations
EMNLP 2022
Neural Differential Equations for Learning to Program Neural Nets Through Continuous Learning Rules
NeurIPS 2022
The Dual Form of Neural Networks Revisited: Connecting Test Time Predictions to Training Patterns via Spotlights of Attention
ICML 2022
The Neural Data Router: Adaptive Control Flow in Transformers Improves Systematic Generalization
ICLR 2022
Going Beyond Linear Transformers with Recurrent Fast Weight Programmers
NeurIPS 2021
Linear Transformers Are Secretly Fast Weight Programmers
ICML 2021
The Devil is in the Detail: Simple Tricks Improve Systematic Generalization of Transformers
EMNLP 2021
Language Modeling with Deep Transformers
Interspeech 2019
On the Choice of Modeling Unit for Sequence-to-Sequence Speech Recognition
Interspeech 2019
RWTH ASR Systems for LibriSpeech: Hybrid vs Attention
Interspeech 2019
Improved Training of End-to-end Attention Models for Speech Recognition
Interspeech 2018
Investigation on Estimation of Sentence Probability by Combining Forward, Backward and Bi-directional LSTM-RNNs
Interspeech 2018
LSTM, GRU, Highway and a Bit of Attention: An Empirical Overview for Language Modeling in Speech Recognition
Interspeech 2016