Papers
Findings of the Third BabyLM Challenge: Accelerating Language Modeling Research with Cognitively Plausible Data
Lucas Charpentier, Leshem Choshen, Ryan Cotterell et al.
Understanding and Enhancing Mamba-Transformer Hybrids for Memory Recall and Language Modeling
Hyunji Lee, Wenhao Yu, Hongming Zhang et al.
Influence-driven Curriculum Learning for Pre-training on Limited Data
Loris Schoenegger, Lukas Thoma, Terra Blevins et al.
Teacher Demonstrations in a BabyLM’s Zone of Proximal Development for Contingent Multi-Turn Interaction
Suchir Salhan, Hongyi Gu, Donya Rooein et al.
Single layer tiny Co4 outpaces GPT-2 and GPT-BERT
Noor Ul Zain, Mohsin Raza Naseem, Ahsan Adeel
Do Syntactic Categories Help in Developmentally Motivated Curriculum Learning for Language Models?
Arzu Burcu Güven, Anna Rogers, Rob Van Der Goot
A Morpheme-Aware Child-Inspired Language Model
Necva Bölücü, Burcu Can
CurLL: A Developmental Framework to Evaluate Continual Learning in Language Models
Pavan Kalyan Tankala, Shubhra Mishra, Satya Lokam et al.
Beyond Repetition: Text Simplification and Curriculum Learning for Data-Constrained Pretraining
Matthew Theodore Roque, Dan John Velasco
What did you say? Generating Child-Directed Speech Questions to Train LLMs
Whitney Poh, Michael Tombolini, Libby Barak
Two ways into the hall of mirrors: Language exposure and lossy memory drive cross-linguistic grammaticality illusions in language models
Kate McCurdy, Katharina Christian, Amelie Seyfried et al.
A Comparison of Elementary Baselines for BabyLM
Rareș Păpușoi, Sergiu Nisioi
Looking to Learn: Token-wise Dynamic Gating for Low-Resource Vision-Language Modelling
Bianca-Mihaela Ganescu, Suchir Salhan, Andrew Caines et al.
Sample-Efficient Language Modeling with Linear Attention and Lightweight Enhancements
Patrick Haller, Jonas Golde, Alan Akbik
BLiSS: Evaluating Bilingual Learner Competence in Second Language Small Language Models
Yuan Gao, Suchir Salhan, Andrew Caines et al.
Exploring smaller batch sizes for a high-performing BabyLM model architecture
Sharid Loáiciga, Eleni Fysikoudi, Asad B. Sayeed
BitMar: Low-Bit Multimodal Fusion with Episodic Memory for Edge Devices
Euhid Aman, Esteban Carlin, Hsing-Kuo Kenneth Pao et al.
What is the Best Sequence Length for BabyLM?
Suchir Salhan, Richard Diehl Martinez, Zébulon Goriely et al.
Design and Analysis of few Million Parameter Transformer-based Language Models trained over a few Million Tokens Dataset
Yen-Che Hsiao, Abhishek Dutta
Large Language Models and Children Have Different Learning Trajectories in Determiner Acquisition
Olivia La Fiandra, Nathalie Fernandez Echeverri, Patrick Shafto et al.
TafBERTa: Learning Grammatical Rules from Small-Scale Language Acquisition Data in Hebrew
Anita Gelboim, Elior Sulem
Model Merging to Maintain Language-Only Performance in Developmentally Plausible Multimodal Models
Ece Takmaz, Lisa Bylinina, Jakub Dotlacil
Are BabyLMs Deaf to Gricean Maxims? A Pragmatic Evaluation of Sample-efficient Language Models
Raha Askari, Sina Zarrieß, Özge Alacam et al.