ICLR 2021

859 papers

ARMOURED: Adversarially Robust MOdels using Unlabeled data by REgularizing Diversity

Kangkang Lu, Cuong Manh Nguyen, Xun Xu, Kiran Krishnamachari, Yu Jing Goh

When does preconditioning help or hurt generalization?

Shun-ichi Amari, Jimmy Ba, Roger Baker Grosse, Xuechen Li, Atsushi Nitanda

Practical Massively Parallel Monte-Carlo Tree Search Applied to Molecular Design

Xiufeng Yang, Tanuj Aasawat, Kazuki Yoshizoe

Rethinking Positional Encoding in Language Pre-training

Guolin Ke, Di He, Tie-yan Liu

Graph Information Bottleneck for Subgraph Recognition

Junchi Yu, Tingyang Xu, Yu Rong, Yatao Bian, Junzhou Huang

i-Mix: A Domain-Agnostic Strategy for Contrastive Representation Learning

Kibok Lee, Yian Zhu, Kihyuk Sohn, Chun-Liang Li, Jinwoo Shin

Few-Shot Bayesian Optimization with Deep Kernel Surrogates

Martin Wistuba, Josif Grabocka

Variational State-Space Models for Localisation and Dense 3D Mapping in 6 DoF

Atanas Mirchev, Baris Kayalibay, Patrick van der Smagt, Justin Bayer

Coping with Label Shift via Distributionally Robust Optimisation

Jingzhao Zhang, Aditya Krishna Menon, Andreas Veit, Srinadh Bhojanapalli, Sanjiv Kumar

Refining Deep Generative Models via Discriminator Gradient Flow

Abdul Fatir Ansari, Ming Liang Ang, Harold Soh

Probing BERT in Hyperbolic Spaces

Boli Chen, Yao Fu, Guangwei Xu, Pengjun Xie, Chuanqi Tan

Calibration of Neural Networks using Splines

Kartik Gupta, Amir Rahimi, Thalaiyasingam Ajanthan, Thomas Mensink, Cristian Sminchisescu

Empirical Analysis of Unlabeled Entity Problem in Named Entity Recognition

Yangming Li, Lemao Liu, Shuming Shi

Repurposing Pretrained Models for Robust Out-of-domain Few-Shot Learning

Namyeong Kwon, Hwidong Na, Gabriel Huang, Simon Lacoste-Julien

Learning advanced mathematical computations from examples

Francois Charton, Amaury Hayat, Guillaume Lample

Learning "What-if" Explanations for Sequential Decision-Making

Ioana Bica, Daniel Jarrett, Alihan Hüyük, Mihaela van der Schaar

Scalable Bayesian Inverse Reinforcement Learning

Alex James Chan, Mihaela van der Schaar

RODE: Learning Roles to Decompose Multi-Agent Tasks

Tonghan Wang, Tarun Gupta, Anuj Mahajan, Bei Peng, Shimon Whiteson

A Trainable Optimal Transport Embedding for Feature Aggregation and its Relationship to Attention

Grégoire Mialon, Dexiong Chen, Alexandre D'aspremont, Julien Mairal

Learning perturbation sets for robust machine learning

Eric Wong, J Zico Kolter

Interpreting Knowledge Graph Relation Representation from Word Embeddings

Carl Allen, Ivana Balazevic, Timothy Hospedales

Deep Networks and the Multiple Manifold Problem

Sam Buchanan, Dar Gilboa, John Wright

Remembering for the Right Reasons: Explanations Reduce Catastrophic Forgetting

Sayna Ebrahimi, Suzanne Petryk, Akash Gokul, William Gan, Joseph E. Gonzalez

Large Associative Memory Problem in Neurobiology and Machine Learning

Dmitry Krotov, John J. Hopfield

Knowledge distillation via softmax regression representation learning

Jing Yang, Brais Martinez, Adrian Bulat, Georgios Tzimiropoulos

OPAL: Offline Primitive Discovery for Accelerating Offline Reinforcement Learning

Anurag Ajay, Aviral Kumar, Pulkit Agrawal, Sergey Levine, Ofir Nachum

Tomographic Auto-Encoder: Unsupervised Bayesian Recovery of Corrupted Data

Francesco Tonolini, Pablo Garcia Moreno, Andreas Damianou, Roderick Murray-Smith

What they do when in doubt: a study of inductive biases in seq2seq learners

Eugene Kharitonov, Rahma Chaabouni

Graph Traversal with Tensor Functionals: A Meta-Algorithm for Scalable Learning

Elan Sopher Markowitz, Keshav Balasubramanian, Mehrnoosh Mirtaheri, Sami Abu-El-Haija, Bryan Perozzi

Multi-Time Attention Networks for Irregularly Sampled Time Series

Satya Narayan Shukla, Benjamin Marlin

Conservative Safety Critics for Exploration

Homanga Bharadhwaj, Aviral Kumar, Nicholas Rhinehart, Sergey Levine, Florian Shkurti

What Can You Learn From Your Muscles? Learning Visual Representation from Human Interactions

Kiana Ehsani, Daniel Gordon, Thomas Hai Dang Nguyen, Roozbeh Mottaghi, Ali Farhadi

Saliency is a Possible Red Herring When Diagnosing Poor Generalization

Joseph D Viviano, Becks Simpson, Francis Dutil, Yoshua Bengio, Joseph Paul Cohen

Bowtie Networks: Generative Modeling for Joint Few-Shot Recognition and Novel-View Synthesis

Zhipeng Bao, Yu-Xiong Wang, Martial Hebert

Implicit Under-Parameterization Inhibits Data-Efficient Deep Reinforcement Learning

Aviral Kumar, Rishabh Agarwal, Dibya Ghosh, Sergey Levine

Heteroskedastic and Imbalanced Deep Learning with Adaptive Regularization

Kaidi Cao, Yining Chen, Junwei Lu, Nikos Arechiga, Adrien Gaidon

Unsupervised Meta-Learning through Latent-Space Interpolation in Generative Models

Siavash Khodadadeh, Sharare Zehtabian, Saeed Vahidian, Weijia Wang, Bill Lin

Trajectory Prediction using Equivariant Continuous Convolution

Robin Walters, Jinxi Li, Rose Yu

BERTology Meets Biology: Interpreting Attention in Protein Language Models

Jesse Vig, Ali Madani, Lav R. Varshney, Caiming Xiong, Richard Socher

VA-RED$^2$: Video Adaptive Redundancy Reduction

Bowen Pan, Rameswar Panda, Camilo Luciano Fosco, Chung-Ching Lin, Alex J Andonian

Zero-shot Synthesis with Group-Supervised Learning

Yunhao Ge, Sami Abu-El-Haija, Gan Xin, Laurent Itti

Integrating Categorical Semantics into Unsupervised Domain Translation

Samuel Lavoie-Marchildon, Faruk Ahmed, Aaron Courville

Selectivity considered harmful: evaluating the causal impact of class selectivity in DNNs

Matthew L Leavitt, Ari S. Morcos

Learning Hyperbolic Representations of Topological Features

Panagiotis Kyriakis, Iordanis Fostiropoulos, Paul Bogdan

Nearest Neighbor Machine Translation

Urvashi Khandelwal, Angela Fan, Dan Jurafsky, Luke Zettlemoyer, Mike Lewis

Stochastic Security: Adversarial Defense Using Long-Run Dynamics of Energy-Based Models

Mitch Hill, Jonathan Craig Mitchell, Song-chun Zhu

Rank the Episodes: A Simple Approach for Exploration in Procedurally-Generated Environments

Daochen Zha, Wenye Ma, Lei Yuan, Xia Hu, Ji Liu

WrapNet: Neural Net Inference with Ultra-Low-Precision Arithmetic

Renkun Ni, Hong-Min Chu, Oscar Castaneda, Ping-yeh Chiang, Christoph Studer

Robust and Generalizable Visual Representation Learning via Random Convolutions

Zhenlin Xu, Deyi Liu, Junlin Yang, Colin Raffel, Marc Niethammer

Enforcing robust control guarantees within neural network policies

Priya L. Donti, Melrose Roderick, Mahyar Fazlyab, J Zico Kolter

DC3: A learning method for optimization with hard constraints

Priya L. Donti, David Rolnick, J Zico Kolter

CO2: Consistent Contrast for Unsupervised Visual Representation Learning

CHEN WEI, Huiyu Wang, Wei Shen, Alan Yuille

Initialization and Regularization of Factorized Neural Layers

Mikhail Khodak, Neil A. Tenenholtz, Lester Mackey, Nicolo Fusi

Anytime Sampling for Autoregressive Models via Ordered Autoencoding

Yilun Xu, Yang Song, Sahaj Garg, Linyuan Gong, Rui Shu

MiCE: Mixture of Contrastive Experts for Unsupervised Image Clustering

Tsung Wei Tsai, Chongxuan Li, Jun Zhu

How to Find Your Friendly Neighborhood: Graph Attention Design with Self-Supervision

Dongkwan Kim, Alice Oh

NeMo: Neural Mesh Models of Contrastive Features for Robust 3D Pose Estimation

Angtian Wang, Adam Kortylewski, Alan Yuille

Combining Physics and Machine Learning for Network Flow Estimation

Arlei Lopes da Silva, Furkan Kocayusufoglu, Saber Jafarpour, Francesco Bullo, Ananthram Swami

Auto Seg-Loss: Searching Metric Surrogates for Semantic Segmentation

Hao Li, Chenxin Tao, Xizhou Zhu, Xiaogang Wang, Gao Huang

Monotonic Kronecker-Factored Lattice

William Taylor Bakst, Nobuyuki Morioka, Erez Louidor

Diverse Video Generation using a Gaussian Process Trigger

Gaurav Shrivastava, Abhinav Shrivastava

On Data-Augmentation and Consistency-Based Semi-Supervised Learning

Atin Ghosh, Alexandre H. Thiery

What Should Not Be Contrastive in Contrastive Learning

Tete Xiao, Xiaolong Wang, Alexei A Efros, Trevor Darrell

GShard: Scaling Giant Models with Conditional Computation and Automatic Sharding

Dmitry Lepikhin, HyoukJoong Lee, Yuanzhong Xu, Dehao Chen, Orhan Firat

Denoising Diffusion Implicit Models

Jiaming Song, Chenlin Meng, Stefano Ermon

NBDT: Neural-Backed Decision Tree

Alvin Wan, Lisa Dunlap, Daniel Ho, Jihan Yin, Scott Lee

Shapley Explanation Networks

Rui Wang, Xiaoqian Wang, David I. Inouye

Learning to Represent Action Values as a Hypergraph on the Action Vertices

Arash Tavakoli, Mehdi Fatemi, Petar Kormushev

Reweighting Augmented Samples by Minimizing the Maximal Expected Loss

Mingyang Yi, Lu Hou, Lifeng Shang, Xin Jiang, Qun Liu

not-MIWAE: Deep Generative Modelling with Missing not at Random Data

Niels Bruun Ipsen, Pierre-Alexandre Mattei, Jes Frellsen

SEDONA: Search for Decoupled Neural Networks toward Greedy Block-wise Learning

Myeongjang Pyeon, Jihwan Moon, Taeyoung Hahn, Gunhee Kim

Entropic gradient descent algorithms and wide flat minima

Fabrizio Pittorino, Carlo Lucibello, Christoph Feinauer, Gabriele Perugini, Carlo Baldassi

The Role of Momentum Parameters in the Optimal Convergence of Adaptive Polyak's Heavy-ball Methods

Wei Tao, Sheng Long, Gaowei Wu, Qing Tao

On the Bottleneck of Graph Neural Networks and its Practical Implications

Uri Alon, Eran Yahav

Auxiliary Learning by Implicit Differentiation

Aviv Navon, Idan Achituve, Haggai Maron, Gal Chechik, Ethan Fetaya

Exploring Balanced Feature Spaces for Representation Learning

Bingyi Kang, Yu Li, Sa Xie, Zehuan Yuan, Jiashi Feng

Unsupervised Discovery of 3D Physical Objects from Video

Yilun Du, Kevin A. Smith, Tomer Ullman, Joshua B. Tenenbaum, Jiajun Wu

DOP: Off-Policy Multi-Agent Decomposed Policy Gradients

Yihan Wang, Beining Han, Tonghan Wang, Heng Dong, Chongjie Zhang

Learning Parametrised Graph Shift Operators

George Dasoulas, Johannes F. Lutzeyer, Michalis Vazirgiannis

Neural Delay Differential Equations

Qunxi Zhu, Yao Guo, Wei LIN

Wandering within a world: Online contextualized few-shot learning

Mengye Ren, Michael Louis Iuzzolino, Michael Curtis Mozer, Richard Zemel

Single-Timescale Actor-Critic Provably Finds Globally Optimal Policy

Zuyue Fu, Zhuoran Yang, Zhaoran Wang

Unsupervised Audiovisual Synthesis via Exemplar Autoencoders

Kangle Deng, Aayush Bansal, Deva Ramanan

Learning with Instance-Dependent Label Noise: A Sample Sieve Approach

Hao Cheng, Zhaowei Zhu, Xingyu Li, Yifei Gong, Xing Sun

Why resampling outperforms reweighting for correcting sampling bias with stochastic gradients

Jing An, Lexing Ying, Yuhua Zhu

R-GAP: Recursive Gradient Attack on Privacy

Junyi Zhu, Matthew B. Blaschko

Flowtron: an Autoregressive Flow-based Generative Network for Text-to-Speech Synthesis

Rafael Valle, Kevin J. Shih, Ryan Prenger, Bryan Catanzaro

Self-Supervised Learning of Compressed Video Representations

Youngjae Yu, Sangho Lee, Gunhee Kim, Yale Song

Few-Shot Learning via Learning the Representation, Provably

Simon Shaolei Du, Wei Hu, Sham M. Kakade, Jason D. Lee, Qi Lei

Influence Functions in Deep Learning Are Fragile

Samyadeep Basu, Phil Pope, Soheil Feizi

Heating up decision boundaries: isocapacitory saturation, adversarial scenarios and generalization bounds

Bogdan Georgiev, Lukas Franken, Mayukh Mukherjee

Fair Mixup: Fairness via Interpolation

Ching-Yao Chuang, Youssef Mroueh

Neural Mechanics: Symmetry and Broken Conservation Laws in Deep Learning Dynamics

Daniel Kunin, Javier Sagastuy-Brena, Surya Ganguli, Daniel LK Yamins, Hidenori Tanaka

Concept Learners for Few-Shot Learning

Kaidi Cao, Maria Brbic, Jure Leskovec

Tradeoffs in Data Augmentation: An Empirical Study

Raphael Gontijo-Lopes, Sylvia Smullin, Ekin Dogus Cubuk, Ethan Dyer

Parameter Efficient Multimodal Transformers for Video Representation Learning

Sangho Lee, Youngjae Yu, Gunhee Kim, Thomas Breuel, Jan Kautz

Risk-Averse Offline Reinforcement Learning

Núria Armengol Urpí, Sebastian Curi, Andreas Krause

Debiasing Concept-based Explanations with Causal Analysis

Mohammad Taha Bahadori, David Heckerman

Contrastive Learning with Hard Negative Samples

Joshua David Robinson, Ching-Yao Chuang, Suvrit Sra, Stefanie Jegelka

Contextual Dropout: An Efficient Sample-Dependent Dropout Module

Xinjie Fan, Shujian Zhang, Korawat Tanwisuth, Xiaoning Qian, Mingyuan Zhou