ICLR 2025

3,703 papers

Large Language Models can Become Strong Self-Detoxifiers

Ching-Yun Ko, Pin-Yu Chen, Payel Das, Youssef Mroueh, Soham Dan

Statistical Tractability of Off-policy Evaluation of History-dependent Policies in POMDPs

Yuheng Zhang, Nan Jiang

Modeling dynamic social vision highlights gaps between deep learning and humans

Kathy Garcia, Emalie McMahon, Colin Conwell, Michael Bonner, Leyla Isik

Teaching LLMs How to Learn with Contextual Fine-Tuning

Younwoo Choi, Muhammad Adil Asif, Ziwen Han, John Willes, Rahul Krishnan

An Undetectable Watermark for Generative Image Models

Sam Gunn, Xuandong Zhao, Dawn Song

Iterative Nash Policy Optimization: Aligning LLMs with General Preferences via No-Regret Learning

Yuheng Zhang, Dian Yu, Baolin Peng, Linfeng Song, Ye Tian

Layer Swapping for Zero-Shot Cross-Lingual Transfer in Large Language Models

Lucas Bandarkar, Benjamin Muller, Pritish Yuvraj, Rui Hou, Nayan Singhal

No Equations Needed: Learning System Dynamics Without Relying on Closed-Form ODEs

Krzysztof Kacprzyk, Mihaela van der Schaar

Learning Continually by Spectral Regularization

Alex Lewandowski, Michał Bortkiewicz, Saurabh Kumar, András György, Dale Schuurmans

Multi-session, multi-task neural decoding from distinct cell-types and brain regions

Mehdi Azabou, Krystal Xuejing Pan, Vinam Arora, Ian Jarratt Knight, Eva L Dyer

Joint Graph Rewiring and Feature Denoising via Spectral Resonance

Jonas Linkerhägner, Cheng Shi, Ivan Dokmanić

Differential learning kinetics govern the transition from memorization to generalization during in-context learning

Alex Nguyen, Gautam Reddy

Learning Diagrams: A Graphical Language for Compositional Training Regimes

Mason Lary, Richard Samuelson, Alexander Wilentz, Alina Zare, Matthew Klawonn

DOTS: Learning to Reason Dynamically in LLMs via Optimal Reasoning Trajectories Search

Murong Yue, Wenlin Yao, Haitao Mi, Dian Yu, Ziyu Yao

Adapters for Altering LLM Vocabularies: What Languages Benefit the Most?

HyoJung Han, Akiko Eriguchi, Haoran Xu, Hieu Hoang, Marine Carpuat

Analyzing Neural Scaling Laws in Two-Layer Networks with Power-Law Data Spectra

Roman Worschech, Bernd Rosenow

PolyPythias: Stability and Outliers across Fifty Language Model Pre-Training Runs

Oskar Van Der Wal, Pietro Lesci, Max Müller-Eberstein, Naomi Saphra, Hailey Schoelkopf

Trained Transformer Classifiers Generalize and Exhibit Benign Overfitting In-Context

Spencer Frei, Gal Vardi

Better autoregressive regression with LLMs via regression-aware fine-tuning

MICHAL LUKASIK, Zhao Meng, Harikrishna Narasimhan, Yin-Wen Chang, Aditya Krishna Menon

P-SPIKESSM: HARNESSING PROBABILISTIC SPIKING STATE SPACE MODELS FOR LONG-RANGE DEPENDENCY TASKS

Malyaban Bal, Abhronil Sengupta

Instruct-SkillMix: A Powerful Pipeline for LLM Instruction Tuning

Simran Kaur, Simon Park, Anirudh Goyal, Sanjeev Arora

Procedural Synthesis of Synthesizable Molecules

Michael Sun, Alston Lo, Minghao Guo, Jie Chen, Connor W. Coley

Dynamic Modeling of Patients, Modalities and Tasks via Multi-modal Multi-task Mixture of Experts

Chenwei Wu, Zitao Shuai, Zhengxu Tang, Luning Wang, Liyue Shen

Policy Design in Long-run Welfare Dynamics

Jiduan Wu, Rediet Abebe, Moritz Hardt, Ana-Andreea Stoica

Mixture of In-Context Prompters for Tabular PFNs

Derek Qiang Xu, F Olcay Cirit, Reza Asadi, Yizhou Sun, Wei Wang

Injective flows for star-like manifolds

Marcello Massimo Negri, Jonathan Aellen, Volker Roth

Turning Up the Heat: Min-p Sampling for Creative and Coherent LLM Outputs

Nguyen Nhat Minh, Andrew Baker, Clement Neo, Allen G Roush, Andreas Kirsch

Adversarial Perturbations Cannot Reliably Protect Artists From Generative AI

Robert Hönig, Javier Rando, Nicholas Carlini, Florian Tramer

Infinite-Resolution Integral Noise Warping for Diffusion Models

Yitong Deng, Winnie Lin, Lingxiao Li, Dmitriy Smirnov, Ryan D Burgert

Context-aware Dynamic Pruning for Speech Foundation Models

Masao Someki, Yifan Peng, Siddhant Arora, Markus Müller, Athanasios Mouchtaris

Behavioral Entropy-Guided Dataset Generation for Offline Reinforcement Learning

Wesley A. Suttle, Aamodh Suresh, Carlos Nieto-Granda

Linear Representations of Political Perspective Emerge in Large Language Models

Junsol Kim, James Evans, Aaron Schein

Diffusing States and Matching Scores: A New Framework for Imitation Learning

Runzhe Wu, Yiding Chen, Gokul Swamy, Kianté Brantley, Wen Sun

Regressing the Relative Future: Efficient Policy Optimization for Multi-turn RLHF

Zhaolin Gao, Wenhao Zhan, Jonathan Daniel Chang, Gokul Swamy, Kianté Brantley

RNNs are not Transformers (Yet): The Key Bottleneck on In-Context Retrieval

Kaiyue Wen, Xingyu Dang, Kaifeng Lyu

MM-EMBED: UNIVERSAL MULTIMODAL RETRIEVAL WITH MULTIMODAL LLMS

Sheng-Chieh Lin, Chankyu Lee, Mohammad Shoeybi, Jimmy Lin, Bryan Catanzaro

Q-SFT: Q-Learning for Language Models via Supervised Fine-Tuning

Joey Hong, Anca Dragan, Sergey Levine

ConceptPrune: Concept Editing in Diffusion Models via Skilled Neuron Pruning

Ruchika Chavhan, Da Li, Timothy Hospedales

Understanding Warmup-Stable-Decay Learning Rates: A River Valley Loss Landscape View

Kaiyue Wen, Zhiyuan Li, Jason S. Wang, David Leo Wright Hall, Percy Liang

Language Models Are Implicitly Continuous

Samuele Marro, Davide Evangelista, X. Angelo Huang, Emanuele La Malfa, Michele Lombardi

Adaptive Batch Size for Privately Finding Second-Order Stationary Points

Daogao Liu, Kunal Talwar

Query-based Knowledge Transfer for Heterogeneous Learning Environments

Norah Alballa, Wenxuan Zhang, Ziquan Liu, Ahmed M. Abdelmoniem, Mohamed Elhoseiny

Revisiting Zeroth-Order Optimization: Minimum-Variance Two-Point Estimators and Directionally Aligned Perturbations

Shaocong Ma, Heng Huang

R-Sparse: Rank-Aware Activation Sparsity for Efficient LLM Inference

Zhenyu Zhang, Zechun Liu, Yuandong Tian, Harshit Khaitan, Zhangyang Wang

Continual Slow-and-Fast Adaptation of Latent Neural Dynamics (CoSFan): Meta-Learning What-How & When to Adapt

Ryan Missel, Linwei Wang

Episodic Memories Generation and Evaluation Benchmark for Large Language Models

Alexis Huet, Zied Ben Houidi, Dario Rossi

Uncovering Latent Memories in Large Language Models

Sunny Duan, Mikail Khona, Abhiram Iyer, Rylan Schaeffer, Ila R Fiete

HELMET: How to Evaluate Long-context Models Effectively and Thoroughly

Howard Yen, Tianyu Gao, Minmin Hou, Ke Ding, Daniel Fleischer

Computationally Efficient RL under Linear Bellman Completeness for Deterministic Dynamics

Runzhe Wu, Ayush Sekhari, Akshay Krishnamurthy, Wen Sun

The Ramanujan Library - Automated Discovery on the Hypergraph of Integer Relations

Itay Beit Halachmi, Ido Kaminer

Holographic Node Representations: Pre-training Task-Agnostic Node Embeddings

Beatrice Bevilacqua, Joshua Robinson, Jure Leskovec, Bruno Ribeiro

Adversarially Robust Anomaly Detection through Spurious Negative Pair Mitigation

Hossein Mirzaei, Mojtaba Nafez, Jafar Habibi, Mohammad Sabokrou, Mohammad Hossein Rohban

Cauchy-Schwarz Regularizers

Sueda Taner, Ziyi Wang, Christoph Studer

A Unified Framework for Forward and Inverse Problems in Subsurface Imaging using Latent Space Translations

Naveen Gupta, Medha Sawhney, Arka Daw, Youzuo Lin, Anuj Karpatne

Towards a learning theory of representation alignment

Francesco Insulla, Shuo Huang, Lorenzo Rosasco

Intelligence at the Edge of Chaos

Shiyang Zhang, Aakash Patel, Syed A Rizvi, Nianchen Liu, Sizhuang He

SV-RAG: LoRA-Contextualizing Adaptation of MLLMs for Long Document Understanding

Jian Chen, Ruiyi Zhang, Yufan Zhou, Tong Yu, Franck Dernoncourt

Efficient Imitation under Misspecification

Nicolas Espinosa-Dice, Sanjiban Choudhury, Wen Sun, Gokul Swamy

Decision Tree Induction Through LLMs via Semantically-Aware Evolution

Tennison Liu, Nicolas Huynh, Mihaela van der Schaar

Cocoon: Robust Multi-Modal Perception with Uncertainty-Aware Sensor Fusion

Minkyoung Cho, Yulong Cao, Jiachen Sun, Qingzhao Zhang, Marco Pavone

Fundamental Limitations on Subquadratic Alternatives to Transformers

Josh Alman, Hantao Yu

Data Shapley in One Training Run

Jiachen T. Wang, Prateek Mittal, Dawn Song, Ruoxi Jia

WildBench: Benchmarking LLMs with Challenging Tasks from Real Users in the Wild

Bill Yuchen Lin, Yuntian Deng, Khyathi Chandu, Abhilasha Ravichander, Valentina Pyatkin

A Deep Generative Learning Approach for Two-stage Adaptive Robust Optimization

Aron Brenner, Rahman Khorramfar, Jennifer Z Sun, Saurabh Amin

Benchmarking Vision Language Model Unlearning via Fictitious Facial Identity Dataset

Yingzi Ma, Jiongxiao Wang, Fei Wang, SIYUAN MA, Jiazhao Li

Self-Improvement in Language Models: The Sharpening Mechanism

Audrey Huang, Adam Block, Dylan J Foster, Dhruv Rohatgi, Cyril Zhang

Graph Neural Networks Gone Hogwild

Olga Solodova, Nick Richardson, Deniz Oktay, Ryan P Adams

The Pitfalls of Memorization: When Memorization Hurts Generalization

Reza Bayat, Mohammad Pezeshki, Elvis DOHMATOB, David Lopez-paz, Pascal Vincent

Benchmarking LLMs' Judgments with No Gold Standard

Shengwei Xu, Yuxuan Lu, Grant Schoenebeck, Yuqing Kong

Implicit Bias of Mirror Flow for Shallow Neural Networks in Univariate Regression

Shuang Liang, Guido Montúfar

SymmCD: Symmetry-Preserving Crystal Generation with Diffusion Models

Daniel Levy, Siba Smarak Panigrahi, Sékou-Oumar Kaba, Qiang Zhu, Kin Long Kelvin Lee

Walk the Talk? Measuring the Faithfulness of Large Language Model Explanations

Katie Matton, Robert Ness, John Guttag, Emre Kiciman

Do Mice Grok? Glimpses of Hidden Progress in Sensory Cortex

Tanishq Kumar, Blake Bordelon, Cengiz Pehlevan, Venkatesh N Murthy, Samuel J. Gershman

See It from My Perspective: How Language Affects Cultural Bias in Image Understanding

Amith Ananthram, Elias Stengel-Eskin, Mohit Bansal, Kathleen McKeown

Aligning Language Models with Demonstrated Feedback

Omar Shaikh, Michelle S. Lam, Joey Hejna, Yijia Shao, Hyundong Justin Cho

Discrete GCBF Proximal Policy Optimization for Multi-agent Safe Optimal Control

Songyuan Zhang, Oswin So, Mitchell Black, Chuchu Fan

InvestESG: A multi-agent reinforcement learning benchmark for studying climate investment as a social dilemma

Xiaoxuan Hou, Jiayi Yuan, Joel Z Leibo, Natasha Jaques

No Location Left Behind: Measuring and Improving the Fairness of Implicit Representations for Earth Data

Daniel Cai, Randall Balestriero

Efficient Diffusion Transformer Policies with Mixture of Expert Denoisers for Multitask Learning

Moritz Reuss, Jyothish Pari, Pulkit Agrawal, Rudolf Lioutikov

Chemistry-Inspired Diffusion with Non-Differentiable Guidance

Yuchen Shen, Chenhao Zhang, Sijie Fu, Chenghui Zhou, Newell Washburn

Wasserstein Distances, Neuronal Entanglement, and Sparsity

Shashata Sawmya, Linghao Kong, Ilia Markov, Dan Alistarh, Nir N Shavit

Capturing the Temporal Dependence of Training Data Influence

Jiachen T. Wang, Dawn Song, James Zou, Prateek Mittal, Ruoxi Jia

Preserving Deep Representations in One-Shot Pruning: A Hessian-Free Second-Order Optimization Framework

Ryan Lucas, Rahul Mazumder

DUET: Decentralized Bilevel Optimization without Lower-Level Strong Convexity

Zhen Qin, Zhuqing Liu, Songtao Lu, Yingbin Liang, Jia Liu

Matrix Product Sketching via Coordinated Sampling

Majid Daliri, Juliana Freire, Danrong Li, Christopher Musco

Scaling LLM Test-Time Compute Optimally Can be More Effective than Scaling Parameters for Reasoning

Charlie Victor Snell, Jaehoon Lee, Kelvin Xu, Aviral Kumar

Boosting Multiple Views for pretrained-based Continual Learning

Quyen Tran, Tung Lam Tran, Khanh Doan, Toan Tran, Dinh Phung

Is Your Video Language Model a Reliable Judge?

Ming Liu, Wensheng Zhang

Approaching Rate-Distortion Limits in Neural Compression with Lattice Transform Coding

Eric Lei, Hamed Hassani, Shirin Saeedi Bidokhti

Inference Scaling for Long-Context Retrieval Augmented Generation

Zhenrui Yue, Honglei Zhuang, Aijun Bai, Kai Hui, Rolf Jagerman

A Meta-Learning Approach to Bayesian Causal Discovery

Anish Dhir, Matthew Ashman, James Requeima, Mark van der Wilk

AdaWM: Adaptive World Model based Planning for Autonomous Driving

Hang Wang, Xin Ye, Feng Tao, Chenbin Pan, Abhirup Mallik

SymDiff: Equivariant Diffusion via Stochastic Symmetrisation

Leo Zhang, Kianoosh Ashouritaklimi, Yee Whye Teh, Rob Cornish

Eliciting Human Preferences with Language Models

Belinda Z. Li, Alex Tamkin, Noah Goodman, Jacob Andreas

Make Haste Slowly: A Theory of Emergent Structured Mixed Selectivity in Feature Learning ReLU Networks

Devon Jarvis, Richard Klein, Benjamin Rosman, Andrew M Saxe

SiReRAG: Indexing Similar and Related Information for Multihop Reasoning

Nan Zhang, Prafulla Kumar Choubey, Alexander Fabbri, Gabriel Bernadett-Shapiro, Rui Zhang

Accelerating Task Generalisation with Multi-Level Skill Hierarchies

Thomas P Cannon, Ozgur Simsek

Improving Graph Neural Networks by Learning Continuous Edge Directions

Seong Ho Pahng, Sahand Hormoz

Extendable and Iterative Structure Learning Strategy for Bayesian Networks

Hamid Kalantari, Russell Greiner, Pouria Ramazi

Lie Algebra Canonicalization: Equivariant Neural Operators under arbitrary Lie Groups

Zakhar Shumaylov, Peter Zaika, James Rowbottom, Ferdia Sherry, Melanie Weber