Papers
Is DPO Superior to PPO for LLM Alignment? A Comprehensive Study
Shusheng Xu, Wei Fu, Jiaxuan Gao et al.
Soft Prompt Recovers Compressed LLMs, Transferably
Zhaozhuo Xu, Zirui Liu, Beidi Chen et al.
Contrastive Preference Optimization: Pushing the Boundaries of LLM Performance in Machine Translation
Haoran Xu, Amr Sharaf, Yunmo Chen et al.
Exploring the LLM Journey from Cognition to Expression with Linear Representations
Yuzi Yan, Jialian Li, Yipin Zhang et al.
Mastering Text-to-Image Diffusion: Recaptioning, Planning, and Generating with Multimodal LLMs
Ling Yang, Zhaochen Yu, Chenlin Meng et al.
Junk DNA Hypothesis: Pruning Small Pre-Trained Weights $\textitIrreversibly$ and $\textitMonotonically$ Impairs “Difficult" Downstream Tasks in LLMs
Lu Yin, Ajay Kumar Jaiswal, Shiwei Liu et al.
Outlier Weighed Layerwise Sparsity (OWL): A Missing Secret Sauce for Pruning LLMs to High Sparsity
Lu Yin, You Wu, Zhenyu Zhang et al.
Collage: Light-Weight Low-Precision Strategy for LLM Training
Tao Yu, Gaurav Gupta, Karthick Gopalswamy et al.
tnGPS: Discovering Unknown Tensor Network Structure Search Algorithms via Large Language Models (LLMs)
Junhua Zeng, Chao Li, Zhun Sun et al.
LQER: Low-Rank Quantization Error Reconstruction for LLMs
Cheng Zhang, Jianyi Cheng, George Anthony Constantinides et al.
CaM: Cache Merging for Memory-efficient LLMs Inference
Yuxin Zhang, Yuxuan Du, Gen Luo et al.
Revisiting Zeroth-Order Optimization for Memory-Efficient LLM Fine-Tuning: A Benchmark
Yihua Zhang, Pingzhi Li, Junyuan Hong et al.
GaLore: Memory-Efficient LLM Training by Gradient Low-Rank Projection
Jiawei Zhao, Zhenyu Zhang, Beidi Chen et al.
Star Attention: Efficient LLM Inference over Long Sequences
Shantanu Acharya, Fei Jia, Boris Ginsburg
A Tale of Two Structures: Do LLMs Capture the Fractal Complexity of Language?
Ibrahim Alabdulmohsin, Andreas Peter Steiner
Aligning LLMs by Predicting Preferences from User Writing Samples
Stéphane Aroca-Ouellette, Natalie Mackraz, Barry-John Theobald et al.
LLMs can see and hear without any training
Kumar Ashutosh, Yossi Gandelsman, Xinlei Chen et al.
Autoformulation of Mathematical Optimization Models Using LLMs
Nicolás Astorga, Tennison Liu, Yuanzhang Xiao et al.
MathConstruct: Challenging LLM Reasoning with Constructive Proofs
Mislav Balunovic, Jasper Dekoninck, Nikola Jovanović et al.
CRANE: Reasoning with constrained LLM generation
Debangshu Banerjee, Tarun Suresh, Shubham Ugare et al.
xLSTM 7B: A Recurrent LLM for Fast and Efficient Inference
Maximilian Beck, Korbinian Pöppel, Phillip Lippe et al.
RocketKV: Accelerating Long-Context LLM Inference via Two-Stage KV Cache Compression
Payman Behnam, Yaosheng Fu, Ritchie Zhao et al.
Puzzle: Distillation-Based NAS for Inference-Optimized LLMs
Akhiad Bercovich, Tomer Ronen, Talor Abramovich et al.
Emergent Misalignment: Narrow finetuning can produce broadly misaligned LLMs
Jan Betley, Daniel Chee Hian Tan, Niels Warncke et al.
Forest-of-Thought: Scaling Test-Time Compute for Enhancing LLM Reasoning
Zhenni Bi, Kai Han, Chuanjian Liu et al.