Ruslan Salakhutdinov

Act2See: Emergent Active Visual Perception for Video Reasoning CVPR 2026 Optimizing Test-Time Compute via Meta Reinforcement Finetuning ICML 2025 Confronting Reward Model Overoptimization with Constrained RLHF ICLR 2024 Contrastive Difference Predictive Coding ICLR 2024 Effective Data Augmentation With Diffusion Models ICLR 2024 HEMM: Holistic Evaluation of Multimodal Foundation Models NeurIPS 2024 Inference via Interpolation: Contrastive Representations Provably Enable Planning and Inference NeurIPS 2024 Manifold Preserving Guided Diffusion ICLR 2024 OmniACT: A Dataset and Benchmark for Enabling Multimodal Generalist Autonomous Agents for Desktop and Web ECCV 2024 Plan-Seq-Learn: Language Model Guided RL for Solving Long Horizon Robotics Tasks ICLR 2024 Situated Instruction Following ECCV 2024 Stabilizing Contrastive RL: Techniques for Robotic Goal Reaching from Offline Data ICLR 2024 Stylus: Automatic Adapter Selection for Diffusion Models NeurIPS 2024 A Connection between One-Step RL and Critic Regularization in Reinforcement Learning ICML 2023 A Simple Approach for Visual Room Rearrangement: 3D Mapping and Semantic Search ICLR 2023 Contrastive Example-Based Control L4DC 2023 Cross-modal Attention Congruence Regularization for Vision-Language Relation Alignment ACL 2023 Factorized Contrastive Learning: Going Beyond Multi-view Redundancy NeurIPS 2023 Generating Images with Multimodal Language Models NeurIPS 2023 Grounding Language Models to Images for Multimodal Inputs and Outputs ICML 2023 Imitating Task and Motion Planning with Visuomotor Transformers CoRL 2023 MultiViz: Towards Visualizing and Understanding Multimodal Models ICLR 2023 MultiZoo and MultiBench: A Standardized Toolkit for Multimodal Deep Learning JMLR 2023 Nano: Nested Human-in-the-Loop Reward Learning for Few-shot Language Model Control ACL 2023 Quantifying & Modeling Multimodal Interactions: An Information Decomposition Framework NeurIPS 2023 Scenario-based Question Answering with Interacting Contextual Properties ICLR 2023 SPRING: Studying Papers and Reasoning to play Games NeurIPS 2023 Conditional Contrastive Learning with Kernel ICLR 2022 ConditionalQA: A Complex Reading Comprehension Dataset with Conditional Answers ACL 2022 Contrastive Learning as Goal-Conditioned Reinforcement Learning NeurIPS 2022 C-Planning: An Automatic Curriculum for Learning Goal-Reaching Tasks ICLR 2022 Don’t Copy the Teacher: Data and Model Challenges in Embodied Dialogue EMNLP 2022 FewNLU: Benchmarking State-of-the-Art Methods for Few-Shot Natural Language Understanding ACL 2022 FILM: Following Instructions in Language with Modular Methods ICLR 2022 Imitating Past Successes can be Very Suboptimal NeurIPS 2022 Learning Weakly-supervised Contrastive Representations ICLR 2022 Mismatched No More: Joint Model-Policy Optimization for Model-Based RL NeurIPS 2022 PACS: A Dataset for Physical Audiovisual Commonsense Reasoning ECCV 2022 Paraphrasing Is All You Need for Novel Object Captioning NeurIPS 2022 Recurrent Model-Free RL Can Be a Strong Baseline for Many POMDPs ICML 2022 The Information Geometry of Unsupervised Reinforcement Learning ICLR 2022 Uncertainty Quantification with Pre-trained Language Models: A Large-Scale Empirical Analysis EMNLP 2022 Zero-shot Transfer Learning within a Heterogeneous Graph via Knowledge Transfer Networks NeurIPS 2022 Accelerating Robotic Reinforcement Learning via Parameterized Action Primitives NeurIPS 2021 Case Study: Deontological Ethics in NLP NAACL 2021 C-Learning: Learning to Achieve Goals via Recursive Classification ICLR 2021 Focused Attention Improves Document-Grounded Generation NAACL 2021 Information Obfuscation of Graph Neural Networks ICML 2021 Instabilities of Offline RL with Pre-Trained Neural Representation ICML 2021 Learning Language and Multimodal Privacy-Preserving Markers of Mood from Mobile Data ACL 2021 Learning To Hallucinate Examples From Extrinsic and Intrinsic Supervision ICCV 2021 Multimodal Speech Summarization Through Semantic Concept Learning Interspeech 2021 Off-Dynamics Reinforcement Learning: Training for Transfer with Domain Classifiers ICLR 2021 On Proximal Policy Optimization’s Heavy-tailed Gradients ICML 2021 Reasoning Over Virtual Knowledge Bases With Open Predicate Relations ICML 2021 Replacing Rewards with Examples: Example-Based Policy Search via Recursive Classification NeurIPS 2021 Robust Predictable Control NeurIPS 2021 SEAL: Self-supervised Embodied Active Learning using Exploration and 3D Consistency NeurIPS 2021 Self-supervised Learning from a Multi-view Perspective ICLR 2021 Self-supervised Representation Learning with Relative Predictive Coding ICLR 2021 StylePTB: A Compositional Benchmark for Fine-grained Controllable Text Style Transfer NAACL 2021 Towards Understanding and Mitigating Social Biases in Language Models ICML 2021 Uncertainty Weighted Actor-Critic for Offline Reinforcement Learning ICML 2021 A Closer Look at Accuracy vs. Robustness NeurIPS 2020 Capsules with Inverted Dot-Product Attention Routing ICLR 2020 Differentiable Reasoning over a Virtual Knowledge Base ICLR 2020 Embodied Multimodal Multitask Learning IJCAI 2020 Exploring Controllable Text Generation Techniques COLING 2020 Harnessing the Power of Infinitely Wide Deep Nets on Small-data Tasks ICLR 2020 Learning To Explore Using Active Neural SLAM ICLR 2020 Multimodal Routing: Improving Local and Global Interpretability of Multimodal Language Analysis EMNLP 2020 Neural Methods for Point-wise Dependency Estimation NeurIPS 2020 Neural Topological SLAM for Visual Navigation CVPR 2020 Object Goal Navigation using Goal-Oriented Semantic Exploration NeurIPS 2020 On Reward-Free Reinforcement Learning with Linear Function Approximation NeurIPS 2020 Planning with General Objective Functions: Going Beyond Total Rewards NeurIPS 2020 Politeness Transfer: A Tag and Generate Approach ACL 2020 Reinforcement Learning with General Value Function Approximation: Provably Efficient Approach via Bounded Eluder Dimension NeurIPS 2020 Rewriting History with Inverse RL: Hindsight Inference for Policy Improvement NeurIPS 2020 Topological Sort for Sentence Ordering ACL 2020 Towards Debiasing Sentence Representations ACL 2020 Weakly-Supervised Reinforcement Learning for Controllable Behavior NeurIPS 2020 AutoLoss: Learning Discrete Schedule for Alternate Optimization ICLR 2019 Deep Gamblers: Learning to Abstain with Portfolio Theory NeurIPS 2019 Deep Neural Networks with Multi-Branch Architectures Are Intrinsically Less Non-Convex AISTATS 2019 Graph Neural Tangent Kernel: Fusing Graph Neural Networks with Graph Kernels NeurIPS 2019 Learning Data Manipulation for Augmentation and Weighting NeurIPS 2019 Learning Factorized Multimodal Representations ICLR 2019 Learning Neural Networks with Adaptive Regularization NeurIPS 2019 Learning Representations from Imperfect Time Series Data via Tensor Rank Regularization ACL 2019 MineRL: A Large-Scale Dataset of Minecraft Demonstrations IJCAI 2019 Mixtape: Breaking the Softmax Bottleneck Efficiently NeurIPS 2019 Multimodal Transformer for Unaligned Multimodal Language Sequences ACL 2019 Multiple Futures Prediction NeurIPS 2019 “My Way of Telling a Story”: Persona based Grounded Story Generation ACL 2019 On Exact Computation with an Infinitely Wide Neural Net NeurIPS 2019 Post Selection Inference with Incomplete Maximum Mean Discrepancy Estimator ICLR 2019 Revisiting LSTM Networks for Semi-Supervised Text Classification via Mixed Objective Function AAAI 2019 Search on the Replay Buffer: Bridging Planning and Reinforcement Learning NeurIPS 2019 Strong and Simple Baselines for Multimodal Utterance Embeddings NAACL 2019 Transformer Dissection: An Unified Understanding for Transformer’s Attention via the Lens of Kernel EMNLP 2019 Transformer-XL: Attentive Language Models beyond a Fixed-Length Context ACL 2019 Video Relationship Reasoning Using Gated Spatio-Temporal Energy Graph CVPR 2019 Worst Cases Policy Gradients CoRL 2019 XLNet: Generalized Autoregressive Pretraining for Language Understanding NeurIPS 2019 Active Neural Localization ICLR 2018 A Generic Approach for Escaping Saddle points AISTATS 2018 Breaking the Softmax Bottleneck: A High-Rank RNN Language Model ICLR 2018 Deep Generative Models with Learnable Knowledge Constraints NeurIPS 2018 Gated Path Planning Networks ICML 2018 GLoMo: Unsupervised Learning of Transferable Relational Graphs NeurIPS 2018 HotpotQA: A Dataset for Diverse, Explainable Multi-hop Question Answering EMNLP 2018 How Many Samples are Needed to Estimate a Convolutional Neural Network? NeurIPS 2018 Investigating the Working of Text Classifiers COLING 2018 Neural Map: Structured Memory for Deep Reinforcement Learning ICLR 2018 Neural Models for Reasoning over Multiple Mentions Using Coreference NAACL 2018 On Unifying Deep Generative Models ICLR 2018 Open Domain Question Answering Using Early Fusion of Knowledge Bases and Text EMNLP 2018 Structured Control Nets for Deep Reinforcement Learning ICML 2018 Style Transfer Through Back-Translation ACL 2018 Transformation Autoregressive Networks ICML 2018 Deep Determinantal Point Process for Large-Scale Multi-Label Classification ICCV 2017 Deep Sets NeurIPS 2017 Gated-Attention Readers for Text Comprehension ACL 2017 Good Semi-supervised Learning That Requires a Bad GAN NeurIPS 2017 Improved Variational Autoencoders for Text Modeling using Dilated Convolutions ICML 2017 Learning Robust Visual-Semantic Embeddings ICCV 2017 Semi-Supervised QA with Generative Domain-Adaptive Nets ACL 2017 Spatially Adaptive Computation Time for Residual Networks CVPR 2017 The More You Know: Using Knowledge Graphs for Image Classification CVPR 2017 Toward Controlled Generation of Text ICML 2017 Architectural Complexity Measures of Recurrent Neural Networks NeurIPS 2016 Deep Kernel Learning AISTATS 2016 Deep Neural Networks with Massive Learned Knowledge EMNLP 2016 Iterative Refinement of the Approximate Posterior for Directed Belief Networks NeurIPS 2016 On Multiplicative Integration with Recurrent Neural Networks NeurIPS 2016 Path-Normalized Optimization of Recurrent Neural Networks with ReLU Activations NeurIPS 2016 Review Networks for Caption Generation NeurIPS 2016 Stochastic Variational Deep Kernel Learning NeurIPS 2016 Accurate and conservative estimates of MRF log-likelihood using reverse annealing AISTATS 2015 Aligning Books and Movies: Towards Story-Like Visual Explanations by Watching Movies and Reading Books ICCV 2015 Learning Wake-Sleep Recurrent Attention Models NeurIPS 2015 Path-SGD: Path-Normalized Optimization in Deep Neural Networks NeurIPS 2015 Predicting Deep Zero-Shot Convolutional Neural Networks Using Textual Descriptions ICCV 2015 segDeepM: Exploiting Segmentation and Context in Deep Neural Networks for Object Detection CVPR 2015 Skip-Thought Vectors NeurIPS 2015 A Multiplicative Model for Learning Distributed Text-Based Attribute Representations NeurIPS 2014 Dropout: A Simple Way to Prevent Neural Networks from Overfitting JMLR 2014 Learning Generative Models with Visual Attention NeurIPS 2014 Multimodal Learning with Deep Boltzmann Machines JMLR 2014 Multimodal Neural Language Models ICML 2014 Annealing between distributions by averaging moments NeurIPS 2013 Discriminative Transfer Learning with Tree-based Priors NeurIPS 2013 Learning Stochastic Feedforward Neural Networks NeurIPS 2013 One-shot learning by inverting a compositional causal process NeurIPS 2013 Tensor Analyzers ICML 2013 The Power of Asymmetry in Binary Hashing NeurIPS 2013 A Better Way to Pretrain Deep Boltzmann Machines NeurIPS 2012 Cardinality Restricted Boltzmann Machines NeurIPS 2012 Domain Adaptation: A Small Sample Statistical Approach AISTATS 2012 Hamming Distance Metric Learning NeurIPS 2012 Matrix reconstruction with the local max norm NeurIPS 2012 Multimodal Learning with Deep Boltzmann Machines NeurIPS 2012 Learning to Learn with Compound HD Models NeurIPS 2011 Learning with the weighted trace-norm under arbitrary sampling distributions NeurIPS 2011 Transfer Learning by Borrowing Examples for Multiclass Object Detection NeurIPS 2011 Collaborative Filtering in a Non-Uniform World: Learning with the Weighted Trace Norm NeurIPS 2010 Efficient Learning of Deep Boltzmann Machines AISTATS 2010 Practical Large-Scale Optimization for Max-norm Regularization NeurIPS 2010 Learning in Markov Random Fields using Tempered Transitions NeurIPS 2009 Modelling Relational Data using Bayesian Clustered Tensor Factorization NeurIPS 2009 Replicated Softmax: an Undirected Topic Model NeurIPS 2009 Evaluating probabilities under high-dimensional latent variable models NeurIPS 2008 Probabilistic Matrix Factorization NeurIPS 2007 Using Deep Belief Nets to Learn Covariance Kernels for Gaussian Processes NeurIPS 2007 Deep Boltzmann Machines AISTATS 2005 Learning a Nonlinear Embedding by Preserving Class Neighbourhood Structure AISTATS 1999