Razvan Pascanu

Round and Round We Go! What makes Rotary Positional Encodings useful? ICLR 2025 Attention as a Hypernetwork ICLR 2025 A Large Recurrent Action Model: xLSTM enables Fast Inference for Robotics Tasks ICML 2025 Softmax is not Enough (for Sharp Size Generalisation) ICML 2025 Discovering modular solutions that generalize compositionally ICLR 2024 Kalman Filter for Online Classification of Non-Stationary Data ICLR 2024 Normalization and effective learning rates in reinforcement learning NIPS 2024 Transformers need glasses! Information over-squashing in language tasks NIPS 2024 Non-Stationary Learning of Neural Networks with Automatic Soft Parameter Reset NIPS 2024 Building on Efficient Foundations: Effective Training of LLMs with Structured Feedforward Layers NIPS 2024 No Representation, No Trust: Connecting Representation, Collapse, and Trust Issues in PPO NIPS 2024 Fine-tuning Reinforcement Learning Models is Secretly a Forgetting Mitigation Problem ICML 2024 Universality of Linear Recurrences Followed by Non-linear Projections: Finite-Width Guarantees and Benefits of Complex Eigenvalues ICML 2024 Improving fine-grained understanding in image-text pre-training ICML 2024 Nevis'22: A Stream of 100 Tasks Sampled from 30 Years of Computer Vision Research JMLR 2023 Deep Reinforcement Learning with Plasticity Injection NIPS 2023 Learning to Modulate pre-trained Models in RL NIPS 2023 The Tunnel Effect: Building Data Representations in Deep Neural Networks NIPS 2023 SemPPL: Predicting Pseudo-Labels for Better Contrastive Representations ICLR 2023 Pre-training via Denoising for Molecular Property Prediction ICLR 2023 Understanding Plasticity in Neural Networks ICML 2023 Resurrecting Recurrent Neural Networks for Long Sequences ICML 2023 Behavior Priors for Efficient Reinforcement Learning JMLR 2022 Disentangling Transfer in Continual Reinforcement Learning NIPS 2022 Wide Neural Networks Forget Less Catastrophically ICML 2022 The CLRS Algorithmic Reasoning Benchmark ICML 2022 Linear Mode Connectivity in Multitask and Continual Learning ICLR 2021 Spectral Normalisation for Deep Reinforcement Learning: An Optimisation Perspective ICML 2021 On the Role of Optimization in Double Descent: A Least Squares Study NIPS 2021 Powerpropagation: A sparsity inducing weight reparameterisation NIPS 2021 Continual World: A Robotic Benchmark For Continual Reinforcement Learning NIPS 2021 Understanding the Role of Training Regimes in Continual Learning NIPS 2020 Top-KAST: Top-K Always Sparse Training NIPS 2020 Stabilizing Transformers for Reinforcement Learning ICML 2020 Improving the Gating Mechanism of Recurrent Neural Networks ICML 2020 Meta-Learning with Warped Gradient Descent ICLR 2020 Pointer Graph Networks NIPS 2020 Functional Regularisation for Continual Learning with Gaussian Processes ICLR 2020 Multiplicative Interactions and Where to Find Them ICLR 2020 Meta-Learning with Latent Embedding Optimization ICLR 2019 Continual Unsupervised Representation Learning NIPS 2019 Distilling Policy Distillation AISTATS 2019 Information asymmetry in KL-regularized RL ICLR 2019 Hyperbolic Attention Networks ICLR 2019 Deep reinforcement learning with relational inductive biases ICLR 2019 Memory-based Parameter Adaptation ICLR 2018 Relational recurrent neural networks NIPS 2018 Progress & Compress: A scalable framework for continual learning ICML 2018 Model compression via distillation and quantization ICLR 2018 Mix & Match Agent Curricula for Reinforcement Learning ICML 2018 Been There, Done That: Meta-Learning with Episodic Recall ICML 2018 Sobolev Training for Neural Networks NIPS 2017 A simple neural network module for relational reasoning NIPS 2017 Distral: Robust multitask reinforcement learning NIPS 2017 Visual Interaction Networks: Learning a Physics Simulator from Video NIPS 2017 Imagination-Augmented Agents for Deep Reinforcement Learning NIPS 2017 Sim-to-Real Robot Learning from Pixels with Progressive Nets CORL 2017 Sharp Minima Can Generalize For Deep Nets ICML 2017 Interaction Networks for Learning about Objects, Relations and Physics NIPS 2016 Natural Neural Networks NIPS 2015 On the Number of Linear Regions of Deep Neural Networks NIPS 2014 Identifying and attacking the saddle point problem in high-dimensional non-convex optimization NIPS 2014 On the difficulty of training recurrent neural networks ICML 2013 Learning Algorithms for the Classification Restricted Boltzmann Machine JMLR 2012 Deep Learners Benefit More from Out-of-Distribution Examples AISTATS 2011