ICLR 2024

2,260 papers

On Diffusion Modeling for Anomaly Detection

Victor Livernoche, Vineet Jain, Yashar Hezaveh, Siamak Ravanbakhsh

Don't Trust: Verify -- Grounding LLM Quantitative Reasoning with Autoformalization

Jin Peng Zhou, Charles E Staats, Wenda Li, Christian Szegedy, Kilian Q Weinberger

Detecting Pretraining Data from Large Language Models

Weijia Shi, Anirudh Ajith, Mengzhou Xia, Yangsibo Huang, Daogao Liu

Sparse MoE with Language Guided Routing for Multilingual Machine Translation

Xinyu Zhao, Xuxi Chen, Yu Cheng, Tianlong Chen

Let's do the time-warp-attend: Learning topological invariants of dynamical systems

Noa Moriel, Matt Ricci, Mor Nitzan

REFACTOR: Learning to Extract Theorems from Proofs

Jin Peng Zhou, Yuhuai Wu, Qiyang Li, Roger Baker Grosse

Learning Adaptive Multiresolution Transforms via Meta-Framelet-based Graph Convolutional Network

Tianze Luo, Zhanfeng Mo, Sinno Jialin Pan

What's In My Big Data?

Yanai Elazar, Akshita Bhagia, Ian Helgi Magnusson, Abhilasha Ravichander, Dustin Schwenk

In-Context Pretraining: Language Modeling Beyond Document Boundaries

Weijia Shi, Sewon Min, Maria Lomeli, Chunting Zhou, Margaret Li

Expressive Losses for Verified Robustness via Convex Combinations

Alessandro De Palma, Rudy R Bunel, Krishnamurthy Dj Dvijotham, M. Pawan Kumar, Robert Stanforth

Two-stage LLM Fine-tuning with Less Specialization and More Generalization

Yihan Wang, Si Si, Daliang Li, MICHAL LUKASIK, Felix Yu

An operator preconditioning perspective on training in physics-informed machine learning

Tim De Ryck, Florent Bonnet, Siddhartha Mishra, Emmanuel de Bézenac

NEFTune: Noisy Embeddings Improve Instruction Finetuning

Neel Jain, Ping-yeh Chiang, Yuxin Wen, John Kirchenbauer, Hong-Min Chu

Graph-based Virtual Sensing from Sparse and Partial Multivariate Observations

Giovanni De Felice, Andrea Cini, Daniele Zambon, Vladimir Gusev, Cesare Alippi

Offline RL with Observation Histories: Analyzing and Improving Sample Complexity

Joey Hong, Anca Dragan, Sergey Levine

Understanding Certified Training with Interval Bound Propagation

Yuhao Mao, Mark Niklas Mueller, Marc Fischer, Martin Vechev

Layer-wise linear mode connectivity

Linara Adilova, Maksym Andriushchenko, Michael Kamp, Asja Fischer, Martin Jaggi

SequenceMatch: Imitation Learning for Autoregressive Sequence Modelling with Backtracking

Chris Cundy, Stefano Ermon

Understanding the Robustness of Multi-modal Contrastive Learning to Distribution Shift

Yihao Xue, Siddharth Joshi, Dang Nguyen, Baharan Mirzasoleiman

Confidential-DPproof: Confidential Proof of Differentially Private Training

Ali Shahin Shamsabadi, Gefei Tan, Tudor Ioan Cebere, Aurélien Bellet, Hamed Haddadi

Jointly-Learned Exit and Inference for a Dynamic Neural Network

Florence Regol, Joud Chataoui, Mark Coates

Improved statistical and computational complexity of the mean-field Langevin dynamics under structured data

Atsushi Nitanda, Kazusato Oko, Taiji Suzuki, Denny Wu

Can Large Language Models Infer Causation from Correlation?

Zhijing Jin, Jiarui Liu, Zhiheng LYU, Spencer Poff, Mrinmaya Sachan

A Variational Perspective on Solving Inverse Problems with Diffusion Models

Morteza Mardani, Jiaming Song, Jan Kautz, Arash Vahdat

Investigating the Benefits of Projection Head for Representation Learning

Yihao Xue, Eric Gan, Jiayi Ni, Siddharth Joshi, Baharan Mirzasoleiman

Sparse Spiking Neural Network: Exploiting Heterogeneity in Timescales for Pruning Recurrent SNN

Biswadeep Chakraborty, Beomseok Kang, Harshit Kumar, Saibal Mukhopadhyay

GNNBoundary: Towards Explaining Graph Neural Networks through the Lens of Decision Boundaries

Xiaoqi Wang, Han Wei Shen

SineNet: Learning Temporal Dynamics in Time-Dependent Partial Differential Equations

Xuan Zhang, Jacob Helwig, Yuchao Lin, Yaochen Xie, Cong Fu

Empirical Analysis of Model Selection for Heterogeneous Causal Effect Estimation

Divyat Mahajan, Ioannis Mitliagkas, Brady Neal, Vasilis Syrgkanis

Towards Identifiable Unsupervised Domain Translation: A Diversified Distribution Matching Approach

Sagar Shrestha, Xiao Fu

Likelihood Training of Cascaded Diffusion Models via Hierarchical Volume-preserving Maps

Henry Li, Ronen Basri, Yuval Kluger

Large Language Models to Enhance Bayesian Optimization

Tennison Liu, Nicolás Astorga, Nabeel Seedat, Mihaela van der Schaar

Circuit Component Reuse Across Tasks in Transformer Language Models

Jack Merullo, Carsten Eickhoff, Ellie Pavlick

On the Variance of Neural Network Training with respect to Test Sets and Distributions

Keller Jordan

DRSM: De-Randomized Smoothing on Malware Classifier Providing Certified Robustness

Shoumik Saha, Wenxiao Wang, Yigitcan Kaya, Soheil Feizi, Tudor Dumitras

Optimistic Bayesian Optimization with Unknown Constraints

Quoc Phong Nguyen, Wan Theng Ruth Chew, Le Song, Bryan Kian Hsiang Low, Patrick Jaillet

VCR-Graphormer: A Mini-batch Graph Transformer via Virtual Connections

Dongqi Fu, Zhigang Hua, Yan Xie, Jin Fang, Si Zhang

To Grok or not to Grok: Disentangling Generalization and Memorization on Corrupted Algorithmic Datasets

Darshil Doshi, Aritra Das, Tianyu He, Andrey Gromov

Efficient Heterogeneous Meta-Learning via Channel Shuffling Modulation

Minh Hoang, Carl Kingsford

Lie Group Decompositions for Equivariant Neural Networks

Mircea Mironenco, Patrick Forré

Diffeomorphic Mesh Deformation via Efficient Optimal Transport for Cortical Surface Reconstruction

Thanh Tung Le, Khai Nguyen, Shanlin Sun, Kun Han, nhật Hồ

The Effectiveness of Random Forgetting for Robust Generalization

Vijaya Raghavan T Ramkumar, Bahram Zonooz, Elahe Arani

Modeling Boundedly Rational Agents with Latent Inference Budgets

Athul Paul Jacob, Abhishek Gupta, Jacob Andreas

Memorization Capacity of Multi-Head Attention in Transformers

Sadegh Mahdavi, Renjie Liao, CHRISTOS THRAMPOULIDIS

Pre-Training Goal-based Models for Sample-Efficient Reinforcement Learning

Haoqi Yuan, Zhancun Mu, Feiyang Xie, Zongqing Lu

Proximal Policy Gradient Arborescence for Quality Diversity Reinforcement Learning

Sumeet Batra, Bryon Tjanaka, Matthew Christopher Fontaine, Aleksei Petrenko, Stefanos Nikolaidis

From Bricks to Bridges: Product of Invariances to Enhance Latent Space Communication

Irene Cannistraci, Luca Moschella, Marco Fumero, Valentino Maiorca, Emanuele Rodolà

Ito Diffusion Approximation of Universal Ito Chains for Sampling, Optimization and Boosting

Aleksei Ustimenko, Aleksandr Beznosikov

Beyond Accuracy: Evaluating Self-Consistency of Code Large Language Models with IdentityChain

Marcus J. Min, Yangruibo Ding, Luca Buratti, Saurabh Pujar, Gail Kaiser

Context-Aware Meta-Learning

Christopher Fifty, Dennis Duan, Ronald Guenther Junkins, Ehsan Amid, Jure Leskovec

Tree Search-Based Policy Optimization under Stochastic Execution Delay

David Valensi, Esther Derman, Shie Mannor, Gal Dalal

Designing Skill-Compatible AI: Methodologies and Frameworks in Chess

Karim Hamade, Reid McIlroy-Young, Siddhartha Sen, Jon Kleinberg, Ashton Anderson

A Flexible Generative Model for Heterogeneous Tabular EHR with Missing Modality

Huan He, William hao, Yuanzhe Xi, Yong Chen, Bradley Malin

Differentiable Learning of Generalized Structured Matrices for Efficient Deep Neural Networks

Changwoo Lee, Hun-Seok Kim

Maximum Likelihood Estimation is All You Need for Well-Specified Covariate Shift

Jiawei Ge, Shange Tang, Jianqing Fan, Cong Ma, Chi Jin

Predictive, scalable and interpretable knowledge tracing on structured domains

Hanqi Zhou, Robert Bamler, Charley M Wu, Álvaro Tejero-Cantero

Directly Fine-Tuning Diffusion Models on Differentiable Rewards

Kevin Clark, Paul Vicol, Kevin Swersky, David J. Fleet

Llemma: An Open Language Model for Mathematics

Zhangir Azerbayev, Hailey Schoelkopf, Keiran Paster, Marco Dos Santos, Stephen Marcus McAleer

Task structure and nonlinearity jointly determine learned representational geometry

Matteo Alleman, Jack Lindsey, Stefano Fusi

Risk Bounds of Accelerated SGD for Overparameterized Linear Regression

Xuheng Li, Yihe Deng, Jingfeng Wu, Dongruo Zhou, Quanquan Gu

Predictive auxiliary objectives in deep RL mimic learning in the brain

Ching Fang, Kim Stachenfeld

A Discretization Framework for Robust Contextual Stochastic Optimization

Rares C Cristian, Georgia Perakis

Variance-aware Regret Bounds for Stochastic Contextual Dueling Bandits

Qiwei Di, Tao Jin, Yue Wu, Heyang Zhao, Farzad Farnoud

MERT: Acoustic Music Understanding Model with Large-Scale Self-supervised Training

Yizhi Li, Ruibin Yuan, Ge Zhang, Yinghao Ma, Xingran Chen

Harnessing Density Ratios for Online Reinforcement Learning

Philip Amortila, Dylan J Foster, Nan Jiang, Ayush Sekhari, Tengyang Xie

MuSR: Testing the Limits of Chain-of-thought with Multistep Soft Reasoning

Zayne Rea Sprague, Xi Ye, Kaj Bostrom, Swarat Chaudhuri, Greg Durrett

Group Preference Optimization: Few-Shot Alignment of Large Language Models

Siyan Zhao, John Dang, Aditya Grover

NeuroBack: Improving CDCL SAT Solving using Graph Neural Networks

Wenxi Wang, Yang Hu, Mohit Tiwari, Sarfraz Khurshid, Kenneth McMillan

BadChain: Backdoor Chain-of-Thought Prompting for Large Language Models

Zhen Xiang, Fengqing Jiang, Zidi Xiong, Bhaskar Ramasubramanian, Radha Poovendran

L2MAC: Large Language Model Automatic Computer for Extensive Code Generation

Samuel Holt, Max Ruiz Luyten, Mihaela van der Schaar

Learning to design protein-protein interactions with enhanced generalization

Anton Bushuiev, Roman Bushuiev, Petr Kouba, Anatolii Filkin, Marketa Gabrielova

When Scaling Meets LLM Finetuning: The Effect of Data, Model and Finetuning Method

Biao Zhang, Zhongtao Liu, Colin Cherry, Orhan Firat

Unified Projection-Free Algorithms for Adversarial DR-Submodular Optimization

Mohammad Pedramfar, Yididiya Y. Nadew, Christopher John Quinn, Vaneet Aggarwal

Does CLIP’s generalization performance mainly stem from high train-test similarity?

Prasanna Mayilvahanan, Thaddäus Wiedemer, Evgenia Rusak, Matthias Bethge, Wieland Brendel

Curriculum reinforcement learning for quantum architecture search under hardware errors

Yash J. Patel, Akash Kundu, Mateusz Ostaszewski, Xavier Bonet-Monroig, Vedran Dunjko

MT-Ranker: Reference-free machine translation evaluation by inter-system ranking

Ibraheem Muhammad Moosa, Rui Zhang, Wenpeng Yin

Forward Learning of Graph Neural Networks

Namyong Park, Xing Wang, Antoine Simoulin, Shuai Yang, Grey Yang

Sufficient conditions for offline reactivation in recurrent neural networks

Nanda H Krishna, Colin Bredenberg, Daniel Levenstein, Blake Aaron Richards, Guillaume Lajoie

Cycle Consistency Driven Object Discovery

Aniket Rajiv Didolkar, Anirudh Goyal, Yoshua Bengio

Beyond Spatio-Temporal Representations: Evolving Fourier Transform for Temporal Graphs

Anson Bastos, Kuldeep Singh, Abhishek Nadgeri, Manish Singh, Toyotaro Suzumura

Provable Compositional Generalization for Object-Centric Learning

Thaddäus Wiedemer, Jack Brady, Alexander Panfilov, Attila Juhos, Matthias Bethge

Learning to Make Adherence-aware Advice

Guanting Chen, Xiaocheng Li, Chunlin Sun, Hanzhao Wang

Demystifying Poisoning Backdoor Attacks from a Statistical Perspective

Ganghua Wang, Xun Xian, Ashish Kundu, Jayanth Srinivasa, Xuan Bi

Improved Techniques for Training Consistency Models

Yang Song, Prafulla Dhariwal

Tensor Programs VI: Feature Learning in Infinite Depth Neural Networks

Greg Yang, Dingli Yu, Chen Zhu, Soufiane Hayou

Soft Robust MDPs and Risk-Sensitive MDPs: Equivalence, Policy Gradient, and Sample Complexity

Runyu Zhang, Yang Hu, Na Li

Overthinking the Truth: Understanding how Language Models Process False Demonstrations

Danny Halawi, Jean-Stanislas Denain, Jacob Steinhardt

Improved Efficiency Based on Learned Saccade and Continuous Scene Reconstruction From Foveated Visual Sampling

Jiayang Liu, Yiming Bu, Daniel Tso, Qinru Qiu

LiDAR: Sensing Linear Probing Performance in Joint Embedding SSL Architectures

Vimal Thilak, Chen Huang, Omid Saremi, Laurent Dinh, Hanlin Goh

Fine-Tuning Language Models for Factuality

Katherine Tian, Eric Mitchell, Huaxiu Yao, Christopher D Manning, Chelsea Finn

CNN Kernels Can Be the Best Shapelets

Eric Qu, Yansen Wang, Xufang Luo, Wenqiang He, Kan Ren

Revisiting the Last-Iterate Convergence of Stochastic Gradient Methods

Zijian Liu, Zhengyuan Zhou

Why is SAM Robust to Label Noise?

Christina Baek, J Zico Kolter, Aditi Raghunathan

GAFormer: Enhancing Timeseries Transformers Through Group-Aware Embeddings

Jingyun Xiao, Ran Liu, Eva L Dyer

Confronting Reward Model Overoptimization with Constrained RLHF

Ted Moskovitz, Aaditya K Singh, DJ Strouse, Tuomas Sandholm, Ruslan Salakhutdinov

Stochastic Gradient Descent for Gaussian Processes Done Right

Jihao Andreas Lin, Shreyas Padhy, Javier Antoran, Austin Tripp, Alexander Terenin

Simple Hierarchical Planning with Diffusion

Chang Chen, Fei Deng, Kenji Kawaguchi, Caglar Gulcehre, Sungjin Ahn

Zero-Shot Robotic Manipulation with Pre-Trained Image-Editing Diffusion Models

Kevin Black, Mitsuhiko Nakamoto, Pranav Atreya, Homer Rich Walke, Chelsea Finn

Language Model Detectors Are Easily Optimized Against

Charlotte Nicks, Eric Mitchell, Rafael Rafailov, Archit Sharma, Christopher D Manning

PILOT: An O(1/K)-Convergent Approach for Policy Evaluation with Nonlinear Function Approximation

Zhuqing Liu, Xin Zhang, Jia Liu, Zhengyuan Zhu, Songtao Lu