Róbert Csordás
16 papers · 2019–2025 · 5 conferences · across top CS/AI conferences
Achievements
Jump to papers ↓+7 more ↓ Show less ↑
🐝 Cross-Pollinator (14) 🌉 Interdisciplinary Bridge 🏃 Academic Marathon (6) 🌍 Conference Polyglot (5) 🌈 Renaissance Researcher (6)
🏃
Academic Marathon
(6)
🌉
Interdisciplinary Bridge
🤝
Dynamic Duo
(10)
🗃️
Keyword Collector
(53)
❓
The Questioner
💎
Century Club
(16)
🔥
Unstoppable
(5)
Conferences
EMNLP (5)
ICLR (4)
ICML (3)
NIPS (3)
ACL (1)
Top co-authors
Keywords
attention mechanism
(3)
recurrent neural network
(3)
mixture of expert
(3)
linear transformer
(2)
universal transformer
(2)
systematic generalization
(2)
language model
(2)
neural network
(2)
language modeling
(2)
few-shot learning
(1)
compositional generalization
(1)
efficient inference
(1)
parameter efficient
(1)
gradient descent
(1)
continual learning
(1)
transformer architecture
(1)
natural language understanding
(1)
reinforcement learning
(1)
sparse model
(1)
hyperparameter tuning
(1)
Papers
Measuring In-Context Computation Complexity via Hidden State Prediction
ICML 2025
MrT5: Dynamic Token Merging for Efficient Byte-level Language Models
ICLR 2025
SwitchHead: Accelerating Transformers with Mixture-of-Experts Attention
NIPS 2024
Recurrent Neural Networks Learn to Store and Generate Sequences using Non-Linear Representations
EMNLP 2024
MoEUT: Mixture-of-Experts Universal Transformers
NIPS 2024
Randomized Positional Encodings Boost Length Generalization of Transformers
ACL 2023
Practical Computational Power of Linear Transformers and Their Recurrent and Self-Referential Extensions
EMNLP 2023
Approximating Two-Layer Feedforward Networks for Efficient Transformers
EMNLP 2023
CTL++: Evaluating Generalization on Never-Seen Compositional Patterns of Known Functions, and Compatibility of Neural Representations
EMNLP 2022
The Neural Data Router: Adaptive Control Flow in Transformers Improves Systematic Generalization
ICLR 2022
The Dual Form of Neural Networks Revisited: Connecting Test Time Predictions to Training Patterns via Spotlights of Attention
ICML 2022
A Modern Self-Referential Weight Matrix That Learns to Modify Itself
ICML 2022
The Devil is in the Detail: Simple Tricks Improve Systematic Generalization of Transformers
EMNLP 2021
Are Neural Nets Modular? Inspecting Functional Modularity Through Differentiable Weight Masks
ICLR 2021
Going Beyond Linear Transformers with Recurrent Fast Weight Programmers
NIPS 2021
Improving Differentiable Neural Computers Through Memory Masking, De-allocation, and Link Distribution Sharpness Control
ICLR 2019