ICLR 2025

3,703 papers

MIND: Math Informed syNthetic Dialogues for Pretraining LLMs

Syeda Nahida Akter, Shrimai Prabhumoye, John Kamalu, Sanjeev Satheesh, Eric Nyberg

LICORICE: Label-Efficient Concept-Based Interpretable Reinforcement Learning

Zhuorui Ye, Stephanie Milani, Geoffrey J. Gordon, Fei Fang

Has the Deep Neural Network learned the Stochastic Process? An Evaluation Viewpoint

Harshit Kumar, Beomseok Kang, Biswadeep Chakraborty, Saibal Mukhopadhyay

Affine Steerable Equivariant Layer for Canonicalization of Neural Networks

Yikang LI, Yeqing Qiu, Yuxuan Chen, Zhouchen Lin

Montessori-Instruct: Generate Influential Training Data Tailored for Student Learning

Xiaochuan Li, Zichun Yu, Chenyan Xiong

What Do You See in Common? Learning Hierarchical Prototypes over Tree-of-Life to Discover Evolutionary Traits

Harish Babu Manogaran, M. Maruf, Arka Daw, Kazi Sajeed Mehrab, Caleb Patrick Charpentier

Model-based RL as a Minimalist Approach to Horizon-Free and Second-Order Bounds

Zhiyong Wang, Dongruo Zhou, John C.S. Lui, Wen Sun

Efficient Active Imitation Learning with Random Network Distillation

Emilien Biré, Anthony Kobanda, Ludovic DENOYER, Rémy Portelas

TabWak: A Watermark for Tabular Diffusion Models

Chaoyi Zhu, Jiayi Tang, Jeroen M. Galjaard, Pin-Yu Chen, Robert Birke

CEB: Compositional Evaluation Benchmark for Fairness in Large Language Models

Song Wang, Peng Wang, Tong Zhou, Yushun Dong, Zhen Tan

ShEPhERD: Diffusing shape, electrostatics, and pharmacophores for bioisosteric drug design

Keir Adams, Kento Abeywardane, Jenna Fromer, Connor W. Coley

How Do Large Language Models Understand Graph Patterns? A Benchmark for Graph Pattern Comprehension

Xinnan Dai, Haohao Qu, Yifei Shen, Bohang Zhang, Qihao Wen

SLMRec: Distilling Large Language Models into Small for Sequential Recommendation

Wujiang Xu, Qitian Wu, Zujie Liang, Jiaojiao Han, Xuying Ning

Resolution Attack: Exploiting Image Compression to Deceive Deep Neural Networks

Wangjia Yu, Xiaomeng Fu, Qiao Li, Jizhong Han, Xiaodan Zhang

Tight Lower Bounds under Asymmetric High-Order Hölder Smoothness and Uniform Convexity

Site Bai, Brian Bullins

Breaking Mental Set to Improve Reasoning through Diverse Multi-Agent Debate

Yexiang Liu, Jie Cao, Zekun Li, Ran He, Tieniu Tan

Discrete Diffusion Schrödinger Bridge Matching for Graph Transformation

Jun Hyeong Kim, SEONGHWAN KIM, Seokhyun Moon, Hyeongwoo Kim, Jeheon Woo

To Clip or not to Clip: the Dynamics of SGD with Gradient Clipping in High-Dimensions

Noah Marshall, Ke Liang Xiao, Atish Agarwala, Elliot Paquette

WardropNet: Traffic Flow Predictions via Equilibrium-Augmented Learning

Kai Jungel, Dario Paccagnan, Axel Parmentier, Maximilian Schiffer

Online Reinforcement Learning in Non-Stationary Context-Driven Environments

Pouya Hamadanian, Arash Nasr-Esfahany, Malte Schwarzkopf, Siddhartha Sen, Mohammad Alizadeh

Efficient Source-Free Time-Series Adaptation via Parameter Subspace Disentanglement

Gaurav Patel, Christopher Michael Sandino, Behrooz Mahasseni, Ellen L. Zippi, Erdrin Azemi

Narrowing Information Bottleneck Theory for Multimodal Image-Text Representations Interpretability

Zhiyu Zhu, Zhibo Jin, Jiayu Zhang, Nan Yang, Jiahao Huang

Logically Consistent Language Models via Neuro-Symbolic Integration

Diego Calanzone, Stefano Teso, Antonio Vergari

Bonsai: Gradient-free Graph Condensation for Node Classification

Mridul Gupta, Samyak Jain, Vansh Ramani, HARIPRASAD KODAMANA, Sayan Ranu

MA-RLHF: Reinforcement Learning from Human Feedback with Macro Actions

Yekun Chai, Haoran Sun, Huang Fang, Shuohuan Wang, Yu Sun

Generalization and Distributed Learning of GFlowNets

Tiago Silva, Amauri H Souza, Omar Rivasplata, Vikas Garg, Samuel Kaski

TexTailor: Customized Text-aligned Texturing via Effective Resampling

Suin Lee, Daeshik Kim

MMTEB: Massive Multilingual Text Embedding Benchmark

Kenneth Enevoldsen, Isaac Chung, Imene Kerboua, Márton Kardos, Ashwin Mathur

Mind the GAP: Glimpse-based Active Perception improves generalization and sample efficiency of visual reasoning

Oleh Kolner, Thomas Ortner, Stanisław Woźniak, Angeliki Pantazi

Talking Turns: Benchmarking Audio Foundation Models on Turn-Taking Dynamics

Siddhant Arora, Zhiyun Lu, Chung-Cheng Chiu, Ruoming Pang, Shinji Watanabe

DAMO: Decoding by Accumulating Activations Momentum for Mitigating Hallucinations in Vision-Language Models

Kaishen Wang, Hengrui Gu, Meijun Gao, Kaixiong Zhou

Real-time design of architectural structures with differentiable mechanics and neural networks

Rafael Pastrana, Eder Medina, Isabel M. de Oliveira, Sigrid Adriaenssens, Ryan P Adams

BEEM: Boosting Performance of Early Exit DNNs using Multi-Exit Classifiers as Experts

Divya Jyoti Bajpai, Manjesh Kumar Hanawal

Scaling Laws for Adversarial Attacks on Language Model Activations and Tokens

Stanislav Fort

Locally Connected Echo State Networks for Time Series Forecasting

Filip Matzner, František Mráz

Reinforcement Learning for Control of Non-Markovian Cellular Population Dynamics

Josiah C Kratz, Jacob Adamczyk

BRIGHT: A Realistic and Challenging Benchmark for Reasoning-Intensive Retrieval

Hongjin Su, Howard Yen, Mengzhou Xia, Weijia Shi, Niklas Muennighoff

Accelerated training through iterative gradient propagation along the residual path

Erwan Fagnou, Paul Caillon, Blaise Delattre, Alexandre Allauzen

Hierarchical Autoregressive Transformers: Combining Byte- and Word-Level Processing for Robust, Adaptable Language Models

Pit Neitemeier, Björn Deiseroth, Constantin Eichenberg, Lukas Balles

U-shaped and Inverted-U Scaling behind Emergent Abilities of Large Language Models

Tung-Yu Wu, Melody Lo

DELIFT: Data Efficient Language model Instruction Fine-Tuning

Ishika Agarwal, Krishnateja Killamsetty, Lucian Popa, Marina Danilevsky

TULIP: Token-length Upgraded CLIP

Ivona Najdenkoska, Mohammad Mahdi Derakhshani, Yuki M Asano, Nanne Van Noord, Marcel Worring

Sketching for Convex and Nonconvex Regularized Least Squares with Sharp Guarantees

Yingzhen Yang, Ping Li

OvercookedV2: Rethinking Overcooked for Zero-Shot Coordination

Tobias Gessler, Tin Dizdarevic, Ani Calinescu, Benjamin Ellis, Andrei Lupu

On the Learn-to-Optimize Capabilities of Transformers in In-Context Sparse Recovery

Renpu Liu, Ruida Zhou, Cong Shen, Jing Yang

Towards Automated Knowledge Integration From Human-Interpretable Representations

Kasia Kobalczyk, Mihaela van der Schaar

AutoDAN-Turbo: A Lifelong Agent for Strategy Self-Exploration to Jailbreak LLMs

Xiaogeng Liu, Peiran Li, G. Edward Suh, Yevgeniy Vorobeychik, Zhuoqing Mao

Causal Representation Learning from Multimodal Biomedical Observations

Yuewen Sun, Lingjing Kong, Guangyi Chen, Loka Li, Gongxu Luo

Learn-by-interact: A Data-Centric Framework For Self-Adaptive Agents in Realistic Environments

Hongjin Su, Ruoxi Sun, Jinsung Yoon, Pengcheng Yin, Tao Yu

Progressive Compression with Universally Quantized Diffusion Models

Yibo Yang, Justus Will, Stephan Mandt

Robotouille: An Asynchronous Planning Benchmark for LLM Agents

Gonzalo Gonzalez-Pumariega, Leong Su Yean, Neha Sunkara, Sanjiban Choudhury

Dataset Distillation via Knowledge Distillation: Towards Efficient Self-Supervised Pre-training of Deep Networks

Siddharth Joshi, Jiayi Ni, Baharan Mirzasoleiman

Self-Attention-Based Contextual Modulation Improves Neural System Identification

Isaac Lin, Tianye Wang, Shang Gao, Tang Shiming, Tai Sing Lee

Can We Trust Embodied Agents? Exploring Backdoor Attacks against Embodied LLM-Based Decision-Making Systems

Ruochen Jiao, Shaoyuan Xie, Justin Yue, Takami Sato, Lixu Wang

Nonconvex Stochastic Optimization under Heavy-Tailed Noises: Optimal Convergence without Gradient Clipping

Zijian Liu, Zhengyuan Zhou

Training Neural Networks as Recognizers of Formal Languages

Alexandra Butoi, Ghazal Khalighinejad, Anej Svete, Josef Valvoda, Ryan Cotterell

Is uniform expressivity too restrictive? Towards efficient expressivity of GNNs

Sammy Khalife, Josué Tonelli-Cueto

Transformers Struggle to Learn to Search

Abulhair Saparov, Srushti Ajay Pawar, Shreyas Pimpalgaonkar, Nitish Joshi, Richard Yuanzhe Pang

Few-Class Arena: A Benchmark for Efficient Selection of Vision Models and Dataset Difficulty Measurement

Bryan Bo Cao, Lawrence O'Gorman, Michael Coss, Shubham Jain

Multi-agent cooperation through learning-aware policy gradients

Alexander Meulemans, Seijin Kobayashi, Johannes von Oswald, Nino Scherrer, Eric Elmoznino

Flow-based Variational Mutual Information: Fast and Flexible Approximations

Caleb Dahlke, Jason Pacheco

ReGen: Generative Robot Simulation via Inverse Design

Phat Tan Nguyen, Tsun-Hsuan Wang, Zhang-Wei Hong, Erfan Aasi, Andrew Silva

Score-based free-form architectures for high-dimensional Fokker-Planck equations

Feng Liu, Faguo Wu, Xiao Zhang

ForecastBench: A Dynamic Benchmark of AI Forecasting Capabilities

Ezra Karger, Houtan Bastani, Chen Yueh-Han, Zachary Jacobs, Danny Halawi

On the Transfer of Object-Centric Representation Learning

Aniket Rajiv Didolkar, Andrii Zadaianchuk, Anirudh Goyal, Michael Curtis Mozer, Yoshua Bengio

Enabling Realtime Reinforcement Learning at Scale with Staggered Asynchronous Inference

Matthew Riemer, Gopeshh Subbaraj, Glen Berseth, Irina Rish

From Models to Microtheories: Distilling a Model's Topical Knowledge for Grounded Question-Answering

Nathaniel Weir, Bhavana Dalvi Mishra, Orion Weller, Oyvind Tafjord, Sam Hornstein

Language Agents Meet Causality -- Bridging LLMs and Causal World Models

John Gkountouras, Matthias Lindemann, Phillip Lippe, Efstratios Gavves, Ivan Titov

Tracing Representation Progression: Analyzing and Enhancing Layer-Wise Similarity

Jiachen Jiang, Jinxin Zhou, Zhihui Zhu

Unlocking Point Processes through Point Set Diffusion

David Lüdke, Enric Rabasseda Raventós, Marcel Kollovieh, Stephan Günnemann

Dynamic Sparse Training versus Dense Training: The Unexpected Winner in Image Corruption Robustness

Boqian Wu, Qiao Xiao, Shunxin Wang, Nicola Strisciuglio, Mykola Pechenizkiy

Probing the Latent Hierarchical Structure of Data via Diffusion Models

Antonio Sclocchi, Alessandro Favero, Noam Itzhak Levi, Matthieu Wyart

SPAM: Spike-Aware Adam with Momentum Reset for Stable LLM Training

Tianjin Huang, Ziquan Zhu, Gaojie Jin, LU LIU, Zhangyang Wang

OpenPRM: Building Open-domain Process-based Reward Models with Preference Trees

Kaiyan Zhang, Jiayuan Zhang, Haoxin Li, Xuekai Zhu, Ermo Hua

IterGen: Iterative Semantic-aware Structured LLM Generation with Backtracking

Shubham Ugare, Rohan Gumaste, Tarun Suresh, Gagandeep Singh, Sasa Misailovic

End-to-end Learning of Gaussian Mixture Priors for Diffusion Sampler

Denis Blessing, Xiaogang Jia, Gerhard Neumann

FlowDec: A flow-based full-band general audio codec with high perceptual quality

Simon Welker, Matthew Le, Ricky T. Q. Chen, Wei-Ning Hsu, Timo Gerkmann

Scaling Stick-Breaking Attention: An Efficient Implementation and In-depth Study

Shawn Tan, Songlin Yang, Aaron Courville, Rameswar Panda, Yikang Shen

Towards Fast, Specialized Machine Learning Force Fields: Distilling Foundation Models via Energy Hessians

Ishan Amin, Sanjeev Raja, Aditi S. Krishnapriyan

Model-Agnostic Knowledge Guided Correction for Improved Neural Surrogate Rollout

Bharat Srikishan, Daniel O'Malley, Mohamed Mehana, Nicholas Lubbers, Nikhil Muralidhar

Meta-Dynamical State Space Models for Integrative Neural Data Analysis

Ayesha Vermani, Josue Nassar, Hyungju Jeon, Matthew Dowling, Il Memming Park

Optimizing Posterior Samples for Bayesian Optimization via Rootfinding

Taiwo Adebiyi, Bach Do, Ruda Zhang

InsightBench: Evaluating Business Analytics Agents Through Multi-Step Insight Generation

Gaurav Sahu, Abhay Puri, Juan A. Rodriguez, Amirhossein Abaskohi, Mohammad Chegini

Measuring Non-Adversarial Reproduction of Training Data in Large Language Models

Michael Aerni, Javier Rando, Edoardo Debenedetti, Nicholas Carlini, Daphne Ippolito

Scalable Extraction of Training Data from Aligned, Production Language Models

Milad Nasr, Javier Rando, Nicholas Carlini, Jonathan Hayase, Matthew Jagielski

CONGO: Compressive Online Gradient Optimization

Jeremy Carleton, Prathik Vijaykumar, Divyanshu Saxena, Dheeraj Narasimha, Srinivas Shakkottai

Support is All You Need for Certified VAE Training

Changming Xu, Debangshu Banerjee, Deepak Vasisht, Gagandeep Singh

GeoLoRA: Geometric integration for parameter efficient fine-tuning

Steffen Schotthöfer, Emanuele Zangrando, Gianluca Ceruti, Francesco Tudisco, Jonas Kusch

Trivialized Momentum Facilitates Diffusion Generative Modeling on Lie Groups

Yuchen Zhu, Tianrong Chen, Lingkai Kong, Evangelos Theodorou, Molei Tao

Bayesian Optimization of Antibodies Informed by a Generative Model of Evolving Sequences

Alan Nawzad Amin, Nate Gruver, Yilun Kuang, Yucen Lily Li, Hunter Elliott

Bayesian Optimization via Continual Variational Last Layer Training

Paul Brunzema, Mikkel Jordahn, John Willes, Sebastian Trimpe, Jasper Snoek

The Last Iterate Advantage: Empirical Auditing and Principled Heuristic Analysis of Differentially Private SGD

Milad Nasr, Thomas Steinke, Borja Balle, Christopher A. Choquette-Choo, Arun Ganesh

Efficient Sparse PCA via Block-Diagonalization

Alberto Del Pia, Dekun Zhou, Yinglun Zhu

Training Free Guided Flow-Matching with Optimal Control

Luran Wang, Chaoran Cheng, Yizhen Liao, Yanru Qu, Ge Liu

Disentangling Representations through Multi-task Learning

Pantelis Vafidis, Aman Bhargava, Antonio Rangel

The Same but Different: Structural Similarities and Differences in Multilingual Language Modeling

Ruochen Zhang, Qinan Yu, Matianyu Zang, Carsten Eickhoff, Ellie Pavlick

Residual Stream Analysis with Multi-Layer SAEs

Tim Lawson, Lucy Farnik, Conor Houghton, Laurence Aitchison

Robust Transfer of Safety-Constrained Reinforcement Learning Agents

Markel Zubia, Thiago D. Simão, Nils Jansen

PortLLM: Personalizing Evolving Large Language Models with Training-Free and Portable Model Patches

Rana Shahroz, Pingzhi Li, Sukwon Yun, Zhenyu Wang, Shahriar Nirjon

STRAP: Robot Sub-Trajectory Retrieval for Augmented Policy Learning

Marius Memmel, Jacob Berg, Bingqing Chen, Abhishek Gupta, Jonathan Francis