ICLR 2024

2,260 papers

SpaCE: The Spatial Confounding Environment

Mauricio Tec, Ana Trisovic, Michelle Audirac, Sophie Mirabai Woodward, Jie Kate Hu

The mechanistic basis of data dependence and abrupt learning in an in-context classification task

Gautam Reddy

Maximum Entropy Model Correction in Reinforcement Learning

Amin Rakhsha, Mete Kemertas, Mohammad Ghavamzadeh, Amir-massoud Farahmand

Fast-ELECTRA for Efficient Pre-training

Chengyu Dong, Liyuan Liu, Hao Cheng, Jingbo Shang, Jianfeng Gao

Causal Modelling Agents: Causal Graph Discovery through Synergising Metadata- and Data-driven Reasoning

Ahmed Abdulaal, adamos hadjivasiliou, Nina Montana-Brown, Tiantian He, Ayodeji Ijishakin

Language Models Represent Space and Time

Wes Gurnee, Max Tegmark

Toward Student-oriented Teacher Network Training for Knowledge Distillation

Chengyu Dong, Liyuan Liu, Jingbo Shang

An Emulator for Fine-tuning Large Language Models using Small Language Models

Eric Mitchell, Rafael Rafailov, Archit Sharma, Chelsea Finn, Christopher D Manning

Pre-training LiDAR-based 3D Object Detectors through Colorization

Tai-Yu Pan, Chenyang Ma, Tianle Chen, Cheng Perng Phoo, Katie Z Luo

Efficient Modulation for Vision Networks

Xu Ma, Xiyang Dai, Jianwei Yang, Bin Xiao, Yinpeng Chen

The Trickle-down Impact of Reward Inconsistency on RLHF

Lingfeng Shen, Sihao Chen, Linfeng Song, Lifeng Jin, Baolin Peng

Federated Q-Learning: Linear Regret Speedup with Low Communication Cost

Zhong Zheng, Fengyu Gao, Lingzhou Xue, Jing Yang

Finite-Time Analysis of On-Policy Heterogeneous Federated Reinforcement Learning

Chenyu Zhang, Han Wang, Aritra Mitra, James Anderson

Training Diffusion Models with Reinforcement Learning

Kevin Black, Michael Janner, Yilun Du, Ilya Kostrikov, Sergey Levine

Doubly Robust Instance-Reweighted Adversarial Training

Daouda Sow, Sen Lin, Zhangyang Wang, Yingbin Liang

Abstractors and relational cross-attention: An inductive bias for explicit relational reasoning in Transformers

Awni Altabaa, Taylor Whittington Webb, Jonathan D. Cohen, John Lafferty

The Hedgehog & the Porcupine: Expressive Linear Attentions with Softmax Mimicry

Michael Zhang, Kush Bhatia, Hermann Kumbong, Christopher Re

Understanding In-Context Learning in Transformers and LLMs by Learning to Learn Discrete Functions

Satwik Bhattamishra, Arkil Patel, Phil Blunsom, Varun Kanade

Transformer-VQ: Linear-Time Transformers via Vector Quantization

Lucas Dax Lingle

FlashFFTConv: Efficient Convolutions for Long Sequences with Tensor Cores

Daniel Y Fu, Hermann Kumbong, Eric Nguyen, Christopher Re

Independent-Set Design of Experiments for Estimating Treatment and Spillover Effects under Network Interference

Chencheng Cai, Xu Zhang, Edoardo Airoldi

Towards Foundational Models for Molecular Learning on Large-Scale Multi-Task Datasets

Dominique Beaini, Shenyang Huang, Joao Alex Cunha, Zhiyi Li, Gabriela Moisescu-Pareja

AMAGO: Scalable In-Context Reinforcement Learning for Adaptive Agents

Jake Grigsby, Linxi Fan, Yuke Zhu

From Molecules to Materials: Pre-training Large Generalizable Models for Atomic Property Prediction

Nima Shoghi, Adeesh Kolluru, John R. Kitchin, Zachary Ward Ulissi, C. Lawrence Zitnick

PlaSma: Procedural Knowledge Models for Language-based Planning and Re-Planning

Faeze Brahman, Chandra Bhagavatula, Valentina Pyatkin, Jena D. Hwang, Xiang Lorraine Li

Are Models Biased on Text without Gender-related Language?

Catarina G Belém, Preethi Seshadri, Yasaman Razeghi, Sameer Singh

Counting Graph Substructures with Graph Neural Networks

Charilaos Kanatsoulis, Alejandro Ribeiro

Reverse Diffusion Monte Carlo

Xunpeng Huang, Hanze Dong, Yifan Hao, Yian Ma, Tong Zhang

Optimal Sketching for Residual Error Estimation for Matrix and Vector Norms

Yi Li, Honghao Lin, David Woodruff

Tractable MCMC for Private Learning with Pure and Gaussian Differential Privacy

Yingyu Lin, Yian Ma, Yu-Xiang Wang, Rachel Emily Redberg, Zhiqi Bu

Chain of Thought Empowers Transformers to Solve Inherently Serial Problems

Zhiyuan Li, Hong Liu, Denny Zhou, Tengyu Ma

The Consensus Game: Language Model Generation via Equilibrium Search

Athul Paul Jacob, Yikang Shen, Gabriele Farina, Jacob Andreas

Davidsonian Scene Graph: Improving Reliability in Fine-grained Evaluation for Text-to-Image Generation

Jaemin Cho, Yushi Hu, Jason Michael Baldridge, Roopal Garg, Peter Anderson

Neural-Symbolic Recursive Machine for Systematic Generalization

Qing Li, Yixin Zhu, Yitao Liang, Ying Nian Wu, Song-chun Zhu

Structural Fairness-aware Active Learning for Graph Neural Networks

Haoyu Han, Xiaorui Liu, Li Ma, MohamadAli Torkamani, Hui LIU

Generalization in diffusion models arises from geometry-adaptive harmonic representations

Zahra Kadkhodaie, Florentin Guth, Eero P Simoncelli, Stephane Mallat

Scaling physics-informed hard constraints with mixture-of-experts

Nithin Chalapathi, Yiheng Du, Aditi S. Krishnapriyan

TEMPO: Prompt-based Generative Pre-trained Transformer for Time Series Forecasting

Defu Cao, Furong Jia, Sercan O Arik, Tomas Pfister, Yixiang Zheng

Exploring the Promise and Limits of Real-Time Recurrent Learning

Kazuki Irie, Anand Gopalakrishnan, Jürgen Schmidhuber

Are Bert Family Good Instruction Followers? A Study on Their Potential And Limitations

Yisheng Xiao, Juntao Li, Zechen Sun, Zechang Li, Qingrong Xia

UNR-Explainer: Counterfactual Explanations for Unsupervised Node Representation Learning Models

Hyunju Kang, Geonhee Han, Hogun Park

Unveiling the Unseen: Identifiable Clusters in Trained Depthwise Convolutional Kernels

Zahra Babaiee, Peyman Kiasari, Daniela Rus, Radu Grosu

CLaM-TTS: Improving Neural Codec Language Model for Zero-Shot Text-to-Speech

Jaehyeon Kim, Keon Lee, Seungjun Chung, Jaewoong Cho

Large Language Models as Automated Aligners for benchmarking Vision-Language Models

Yuanfeng Ji, Chongjian GE, Weikai Kong, Enze Xie, Zhengying Liu

On the Role of Discrete Tokenization in Visual Representation Learning

Tianqi Du, Yifei Wang, Yisen Wang

On the Scalability and Memory Efficiency of Semidefinite Programs for Lipschitz Constant Estimation of Neural Networks

Zi Wang, Bin Hu, Aaron J Havens, Alexandre Araujo, Yang Zheng

Graph Transformers on EHRs: Better Representation Improves Downstream Performance

Raphael Poulain, Rahmatollah Beheshti

Efficient Inverse Multiagent Learning

Denizalp Goktas, Amy Greenwald, Sadie Zhao, Alec Koppel, Sumitra Ganesh

Generative Adversarial Equilibrium Solvers

Denizalp Goktas, David C. Parkes, Ian Gemp, Luke Marris, Georgios Piliouras

Uncertainty-aware Graph-based Hyperspectral Image Classification

Linlin Yu, Yifei Lou, Feng Chen

Realistic Evaluation of Semi-supervised Learning Algorithms in Open Environments

Lin-Han Jia, Lan-Zhe Guo, Zhi Zhou, Yu-feng Li

sRGB Real Noise Modeling via Noise-Aware Sampling with Normalizing Flows

Dongjin Kim, Donggoo Jung, Sungyong Baik, Tae Hyun Kim

Pushing Boundaries: Mixup's Influence on Neural Collapse

Quinn LeBlanc Fisher, Haoming Meng, Vardan Papyan

Rotation Has Two Sides: Evaluating Data Augmentation for Deep One-class Classification

Guodong Wang, Yunhong Wang, Xiuguo Bao, Di Huang

Enhancing Instance-Level Image Classification with Set-Level Labels

Renyu Zhang, Aly A Khan, Yuxin Chen, Robert L. Grossman

Bias Runs Deep: Implicit Reasoning Biases in Persona-Assigned LLMs

Shashank Gupta, Vaishnavi Shrivastava, Ameet Deshpande, Ashwin Kalyan, Peter Clark

Repeated Random Sampling for Minimizing the Time-to-Accuracy of Learning

Patrik Okanovic, Roger Waleffe, Vasilis Mageirakos, Konstantinos Nikolakakis, Amin Karbasi

PromptAgent: Strategic Planning with Language Models Enables Expert-level Prompt Optimization

Xinyuan Wang, Chenxi Li, Zhen Wang, Fan Bai, Haotian Luo

Be Aware of the Neighborhood Effect: Modeling Selection Bias under Interference

Haoxuan Li, Chunyuan Zheng, Sihao Ding, Peng Wu, Zhi Geng

Dichotomy of Early and Late Phase Implicit Biases Can Provably Induce Grokking

Kaifeng Lyu, Jikai Jin, Zhiyuan Li, Simon Shaolei Du, Jason D. Lee

In-Context Learning Dynamics with Random Binary Sequences

Eric J Bigelow, Ekdeep Singh Lubana, Robert P. Dick, Hidenori Tanaka, Tomer Ullman

Gen-Z: Generative Zero-Shot Text Classification with Contextualized Label Descriptions

Sachin Kumar, Chan Young Park, Yulia Tsvetkov

RECOMP: Improving Retrieval-Augmented LMs with Context Compression and Selective Augmentation

Fangyuan Xu, Weijia Shi, Eunsol Choi

Leftover Lunch: Advantage-based Offline Reinforcement Learning for Language Models

Ashutosh Baheti, Ximing Lu, Faeze Brahman, Ronan Le Bras, Maarten Sap

ArchLock: Locking DNN Transferability at the Architecture Level with a Zero-Cost Binary Predictor

Tong Zhou, Shaolei Ren, Xiaolin Xu

A Quadratic Synchronization Rule for Distributed Deep Learning

Xinran Gu, Kaifeng Lyu, Sanjeev Arora, Jingzhao Zhang, Longbo Huang

SKILL-MIX: a Flexible and Expandable Family of Evaluations for AI Models

Dingli Yu, Simran Kaur, Arushi Gupta, Jonah Brown-Cohen, Anirudh Goyal

Pessimistic Nonlinear Least-Squares Value Iteration for Offline Reinforcement Learning

Qiwei Di, Heyang Zhao, Jiafan He, Quanquan Gu

Learning to Relax: Setting Solver Parameters Across a Sequence of Linear System Instances

Mikhail Khodak, Edmond Chow, Maria Florina Balcan, Ameet Talwalkar

Data Debugging with Shapley Importance over Machine Learning Pipelines

Bojan Karlaš, David Dao, Matteo Interlandi, Sebastian Schelter, Wentao Wu

Meta-VBO: Utilizing Prior Tasks in Optimizing Risk Measures with Gaussian Processes

Quoc Phong Nguyen, Bryan Kian Hsiang Low, Patrick Jaillet

Extending Power of Nature from Binary to Real-Valued Graph Learning in Real World

Chunshu Wu, Ruibing Song, Chuan Liu, Yunan Yang, Ang Li

BooookScore: A systematic exploration of book-length summarization in the era of LLMs

Yapei Chang, Kyle Lo, Tanya Goyal, Mohit Iyyer

FedDA: Faster Adaptive Gradient Methods for Federated Constrained Optimization

Junyi Li, Feihu Huang, Heng Huang

PINNACLE: PINN Adaptive ColLocation and Experimental points selection

Gregory Kang Ruey Lau, Apivich Hemachandra, See-Kiong Ng, Bryan Kian Hsiang Low

What Matters to You? Towards Visual Representation Alignment for Robot Learning

Thomas Tian, Chenfeng Xu, Masayoshi Tomizuka, Jitendra Malik, Andrea Bajcsy

Deceptive Fairness Attacks on Graphs via Meta Learning

Jian Kang, Yinglong Xia, Ross Maciejewski, Jiebo Luo, Hanghang Tong

Energy-based Automated Model Evaluation

Ru Peng, Heming Zou, Haobo Wang, Yawen Zeng, Zenan Huang

Policy Rehearsing: Training Generalizable Policies for Reinforcement Learning

Chengxing Jia, Chenxiao Gao, Hao Yin, Fuxiang Zhang, Xiong-Hui Chen

Chain-of-Knowledge: Grounding Large Language Models via Dynamic Knowledge Adapting over Heterogeneous Sources

Xingxuan Li, Ruochen Zhao, Yew Ken Chia, Bosheng Ding, Shafiq Joty

Improving Generalization of Alignment with Human Preferences through Group Invariant Learning

Rui Zheng, Wei Shen, Yuan Hua, Wenbin Lai, Shihan Dou

LitCab: Lightweight Language Model Calibration over Short- and Long-form Responses

Xin Liu, Muhammad Khalifa, Lu Wang

Assessing Uncertainty in Similarity Scoring: Performance & Fairness in Face Recognition

Jean-Rémy Conti, Stéphan Clémençon

Selective Visual Representations Improve Convergence and Generalization for Embodied AI

Ainaz Eftekhar, Kuo-Hao Zeng, Jiafei Duan, Ali Farhadi, Aniruddha Kembhavi

Kosmos-G: Generating Images in Context with Multimodal Large Language Models

Xichen Pan, Li Dong, Shaohan Huang, Zhiliang Peng, Wenhu Chen

Neural Spectral Methods: Self-supervised learning in the spectral domain

Yiheng Du, Nithin Chalapathi, Aditi S. Krishnapriyan

Retrieval meets Long Context Large Language Models

Peng Xu, Wei Ping, Xianchao Wu, Lawrence McAfee, Chen Zhu

SuRe: Summarizing Retrievals using Answer Candidates for Open-domain QA of LLMs

Jaehyung Kim, Jaehyun Nam, Sangwoo Mo, Jongjin Park, Sang-Woo Lee

The Effective Horizon Explains Deep RL Performance in Stochastic Environments

Cassidy Laidlaw, Banghua Zhu, Stuart Russell, Anca Dragan

Harnessing Explanations: LLM-to-LM Interpreter for Enhanced Text-Attributed Graph Representation Learning

Xiaoxin He, Xavier Bresson, Thomas Laurent, Adam Perold, Yann LeCun

Seeking Neural Nuggets: Knowledge Transfer in Large Language Models from a Parametric Perspective

Ming Zhong, Chenxin An, Weizhu Chen, Jiawei Han, Pengcheng He

RepoBench: Benchmarking Repository-Level Code Auto-Completion Systems

Tianyang Liu, Canwen Xu, Julian McAuley

Mechanistically analyzing the effects of fine-tuning on procedurally defined tasks

Samyak Jain, Robert Kirk, Ekdeep Singh Lubana, Robert P. Dick, Hidenori Tanaka

Diagnosing Transformers: Illuminating Feature Spaces for Clinical Decision-Making

Aliyah R. Hsu, Yeshwanth Cherapanamjeri, Briton Park, Tristan Naumann, Anobel Odisho

A Good Learner can Teach Better: Teacher-Student Collaborative Knowledge Distillation

Ayan Sengupta, Shantanu Dixit, Md Shad Akhtar, Tanmoy Chakraborty

Addressing Loss of Plasticity and Catastrophic Forgetting in Continual Learning

Mohamed Elsayed, A. Rupam Mahmood

On Representation Complexity of Model-based and Model-free Reinforcement Learning

Hanlin Zhu, Baihe Huang, Stuart Russell

Safe Collaborative Filtering

Riku Togashi, Tatsushi Oka, Naoto Ohsaka, Tetsuro Morimura

GraphChef: Decision-Tree Recipes to Explain Graph Neural Networks

Peter Müller, Lukas Faber, Karolis Martinkus, Roger Wattenhofer

Chain of Hindsight aligns Language Models with Feedback

Hao Liu, Carmelo Sferrazza, Pieter Abbeel