Papers
Emergent Convergence in Multi-Agent LLM Annotation
Angelina Parfenova, Alexander Denzler, Jürgen Pfeffer
Understanding the Side Effects of Rank-One Knowledge Editing
Ryosuke Takahashi, Go Kamoda, Benjamin Heinzerling et al.
When LRP Diverges from Leave-One-Out in Transformers
Weiqiu You, Siqi Zeng, Yao-Hung Hubert Tsai et al.
Investigating ReLoRA: Effects on the Learning Dynamics of Small Language Models
Yuval Weiss, David Demitri Africa, Paula Buttery et al.
Interpreting Language Models Through Concept Descriptions: A Survey
Nils Feldhus, Laura Kopf
Language Dominance in Multilingual Large Language Models
Nadav Shani, Ali Basirat
Not a nuisance but a useful heuristic: Outlier dimensions favor frequent tokens in language models
Iuri Macocco, Nora Graichen, Gemma Boleda et al.
The Comparative Trap: Pairwise Comparisons Amplifies Biased Preferences of LLM Evaluators
Hawon Jeong, ChaeHun Park, Jimin Hong et al.
Evil twins are not that evil: Qualitative insights into machine-generated prompts
Nathanaël Carraz Rakotonirina, Corentin Kervadec, Francesca Franzon et al.
Char-mander Use mBackdoor! A Study of Cross-lingual Backdoor Attacks in Multilingual LLMs
Himanshu Beniwal, Sailesh Panda, Birudugadda Srivibhav et al.
CE-Bench: Towards a Reliable Contrastive Evaluation Benchmark of Interpretability of Sparse Autoencoders
Alex Gulko, Yusen Peng, Sachin Kumar
Babies Learn to Look Ahead: Multi-Token Prediction in Small LMs
Ansar Aynetdinov, Alan Akbik
RecombiText: Compositional Data Augmentation for Enhancing LLM Pre-Training Datasets in Low-Resource Scenarios
Alexander Tampier, Lukas Thoma, Loris Schoenegger et al.
MoEP: Modular Expert Paths for Sample-Efficient Language Modeling
Joonas Tapaninaho
Masked Diffusion Language Models with Frequency-Informed Training
Despoina Kosmopoulou, Efthymios Georgiou, Vaggelis Dorovatas et al.
Pretraining Language Models with LoRA and Artificial Languages
Nalin Kumar, Mateusz Lango, Ondrej Dusek
Batch-wise Convergent Pre-training: Step-by-Step Learning Inspired by Child Language Development
Ko Yoshida, Daiki Shiono, Kai Sato et al.
Linguistic Units as Tokens: Intrinsic and Extrinsic Evaluation with BabyLM
Achille Fusco, Maria Letizia Piccini Bianchessi, Tommaso Sgrizzi et al.
Active Curriculum Language Modeling over a Hybrid Pre-training Method
Eleni Fysikoudi, Sharid Loáiciga, Asad B. Sayeed
You are an LLM teaching a smaller model everything you know: Multi-task pretraining of language models with LLM-designed study plans
Wiktor Kamzela, Mateusz Lango, Ondrej Dusek
Once Upon a Time: Interactive Learning for Storytelling with Small Language Models
Jonas Mayer Martins, Ali Hamza Bashir, Muhammad Rehan Khalid et al.
Mask and You Shall Receive: Optimizing Masked Language Modeling For Pretraining BabyLMs
Lukas Edman, Alexander Fraser
CLASS-IT: Conversational and Lecture-Aligned Small-Scale Instruction Tuning for BabyLMs
Luca Capone, Alessandro Bondielli, Alessandro Lenci
Dialogue Is Not Enough to Make a Communicative BabyLM (But Neither Is Developmentally Inspired Reinforcement Learning)
Francesca Padovani, Bastian Bunzeck, Manar Ali et al.