CoNLL 2024

69 papers

AntLM: Bridging Causal and Masked Language Models

Xinru Yu, Bin Guo, Shiwei Luo, Jie Wang, Tao Ji

BERTtime Stories: Investigating the Role of Synthetic Story Data in Language Pre-training

Nikitas Theodoropoulos, Giorgos Filandrianos, Vassilis Lyberatos, Maria Lymperaiou, Giorgos Stamou

Teaching Tiny Minds: Exploring Methods to Enhance Knowledge Distillation for Small Language Models

Hong Meng Yam, Nathan Paek

BabyLlama-2: Ensemble-Distilled Models Consistently Outperform Teachers With Limited Data

Jean-Loup Tastet, Inar Timiryasov

What should Baby Models read? Exploring Sample-Efficient Data Composition on Model Performance

Hong Meng Yam, Nathan Paek

GPT or BERT: why not both?

Lucas Georges Gabriel Charpentier, David Samuel

BabyLM Challenge: Exploring the effect of variation sets on language model training efficiency

Akari Haga, Akiyo Fukatsu, Miyu Oba, Arianna Bisazza, Yohei Oseki

Dreaming Out Loud: A Self-Synthesis Approach For Training Vision-Language Models With Developmentally Plausible Data

Badr AlKhamissi, Yingtian Tang, Abdülkadir Gökce, Johannes Mehrer, Martin Schrimpf

A surprisal oracle for when every layer counts

Xudong Hong, Sharid Loáiciga, Asad Sayeed

WhatIf: Leveraging Word Vectors for Small-Scale Data Augmentation

Alex Lyman, Bryce Hepner

Using Curriculum Masking Based on Child Language Development to Train a Large Language Model with Limited Training Data

Evan Lucas, Dylan Gaines, Tagore Rao Kosireddy, Kevin Li, Timothy C. Havens

Automatic Quality Estimation for Data Selection and Curriculum Learning

Hiep Nguyen, Lynn Yip, Justin DeBenedetto

When Babies Teach Babies: Can student knowledge sharing outperform Teacher-Guided Distillation on small datasets?

Srikrishna Iyer

ConcreteGPT: A Baby GPT-2 Based on Lexical Concreteness and Curriculum Learning

Luca Capone, Alessandro Bondielli, Alessandro Lenci

Less is More: Pre-Training Cross-Lingual Small-Scale Language Models with Cognitively-Plausible Curriculum Learning Strategies

Suchir Salhan, Richard Diehl Martinez, Zébulon Goriely, Paula Buttery

Are BabyLMs Second Language Learners?

Lukas Edman, Lisa Bylinina, Faeze Ghorbanpour, Alexander Fraser

Integrating Quasi-symbolic Conceptual Knowledge into Language Model Pre-training

Gábor Berend

Extending the BabyLM Initiative : Promoting Diversity in Datasets and Metrics through High-Quality Linguistic Corpora

Laurent Prevot, Sheng-Fu Wang, Jou-An Chi, Shu-Kai Hsieh

ELC-ParserBERT: Low-Resource Language Modeling Utilizing a Parser Network With ELC-BERT

Rufus Behr

Developmentally Plausible Multimodal Language Models Are Highly Modular

Alina Klerings, Christian Bartelt, Aaron Mueller

Different Ways to Forget: Linguistic Gates in Recurrent Neural Networks

Cristiano Chesi, Veronica Bressan, Matilde Barbini, Achille Fusco, Maria Letizia Piccini Bianchessi

Choosy Babies Need One Coach: Inducing Mode-Seeking Behavior in BabyLlama with Reverse KL Divergence

Shaozhen Shi, Yevgen Matusevych, Malvina Nissim

BabyHGRN: Exploring RNNs for Sample-Efficient Language Modeling

Patrick Haller, Jonas Golde, Alan Akbik

Exploring Curriculum Learning for Vision-Language Tasks: A Study on Small-Scale Multimodal Training

Rohan Saha, Abrar Fahim, Alona Fyshe, Alex Murphy

Graphemes vs. phonemes: battling it out in character-based language models

Bastian Bunzeck, Daniel Duran, Leonie Schade, Sina Zarrieß

From Babble to Words: Pre-Training Language Models on Continuous Streams of Phonemes

Zébulon Goriely, Richard Diehl Martinez, Andrew Caines, Paula Buttery, Lisa Beinborn

BabyLM Challenge: Experimenting with Self-Distillation and Reverse-Distillation for Language Model Pre-Training on Constrained Datasets

Aakarsh Nair, Alina Hancharova, Mayank Kumar, Ali Gharaee

Towards Data-Efficient Language Models: A Child-Inspired Approach to Language Learning

Mohammad Amin Ghanizadeh, Mohammad Javad Dousti

Findings of the Second BabyLM Challenge: Sample-Efficient Pretraining on Developmentally Plausible Corpora

Michael Y. Hu, Aaron Mueller, Candace Ross, Adina Williams, Tal Linzen

Leveraging a Cognitive Model to Measure Subjective Similarity of Human and GPT-4 Written Content

Tyler Malloy, Maria José Ferreira, Fei Fang, Cleotilde Gonzalez

Advancing Arabic Sentiment Analysis: ArSen Benchmark and the Improved Fuzzy Deep Hybrid Network

Yang Fang, Cheng Xu, Shuhao Guan, Nan Yan, Yuke Mei

Solving the Challenge Set without Solving the Task: On Winograd Schemas as a Test of Pronominal Coreference Resolution

Ian Porada, Jackie CK Cheung

CrowdCounter: A benchmark type-specific multi-target counterspeech dataset

Punyajoy Saha, Abhilash Datta, Abhik Jana, Animesh Mukherjee

One-Vs-Rest Neural Network English Grapheme Segmentation: A Linguistic Perspective

Samuel Rose, Nina Dethlefs, C. Kambhampati

Self-supervised speech representations display some human-like cross-linguistic perceptual abilities

Joselyn Rodriguez, Kamala Sreepada, Ruolan Leslie Famularo, Sharon Goldwater, Naomi Feldman

Image-conditioned human language comprehension and psychometric benchmarking of visual language models

Subha Nawer Pushpita, Roger P. Levy

PRACT: Optimizing Principled Reasoning and Acting of LLM Agent

Zhiwei Liu, Weiran Yao, Jianguo Zhang, Rithesh Murthy, Liangwei Yang

A Multimodal Large Language Model “Foresees” Objects Based on Verb Information but Not Gender

Shuqi Wang, Xufeng Duan, Zhenguang Cai

Explaining the Hardest Errors of Contextual Embedding Based Classifiers

Claudio Moisés Valiense De Andrade, Washington Cunha, Guilherme Fonseca, Ana Clara Souza Pagano, Luana De Castro Santos

Translating Across Cultures: LLMs for Intralingual Cultural Adaptation

Pushpdeep Singh, Mayur Patidar, Lovekesh Vig

Further Compressing Distilled Language Models via Frequency-aware Partial Sparse Coding of Embeddings

Kohki Tamura, Naoki Yoshinaga, Masato Neishi

How Are Metaphors Processed by Language Models? The Case of Analogies

Joanne Boisson, Asahi Ushio, Hsuvas Borkakoty, Kiamehr Rezaee, Dimosthenis Antypas

Text2Afford: Probing Object Affordance Prediction abilities of Language Models solely from Text

Sayantan Adak, Daivik Agrawal, Animesh Mukherjee, Somak Aditya

Aligning Alignments: Do Colexification and Distributional Similarity Align as Measures of cross-lingual Lexical Alignment?

Taelin Karidi, Eitan Grossman, Omri Abend

Continuous Attentive Multimodal Prompt Tuning for Few-Shot Multimodal Sarcasm Detection

Soumyadeep Jana, Animesh Dey, Ranbir Singh Sanasam

Large Language Model Recall Uncertainty is Modulated by the Fan Effect

Jesse Roberts, Kyle Moore, Douglas Fisher, Oseremhen Ewaleifoh, Thao Pham

Is Structure Dependence Shaped for Efficient Communication?: A Case Study on Coordination

Kohei Kajikawa, Yusuke Kubota, Yohei Oseki

Of Models and Men: Probing Neural Networks for Agreement Attraction with Psycholinguistic Data

Maxim Bazhukov, Ekaterina Voloshina, Sergey Pletenev, Arseny Anisimov, Oleg Serikov

Generalizations across filler-gap dependencies in neural language models

Katherine Howitt, Sathvik Nair, Allison Dods, Robert Melvin Hopkins

A Novel Instruction Tuning Method for Vietnamese Mathematical Reasoning using Trainable Open-Source Large Language Models

Quang-Vinh Nguyen, Thanh-Do Nguyen, Van-Vinh Nguyen, Khac-Hoai Nam Bui

NeLLCom-X: A Comprehensive Neural-Agent Framework to Simulate Language Learning and Group Communication

Yuchen Lian, Tessa Verhoef, Arianna Bisazza

Revisiting Hierarchical Text Classification: Inference and Metrics

Roman Plaud, Matthieu Labeau, Antoine Saillenfest, Thomas Bonald

The Effect of Surprisal on Reading Times in Information Seeking and Repeated Reading

Keren Gruteke Klein, Yoav Meiri, Omer Shubi, Yevgeni Berzak

TpT-ADE: Transformer Based Two-Phase ADE Extraction

Suryamukhi Kuchibhotla, Manish Singh

Global Learning with Triplet Relations in Abstractive Summarization

Fengyu Lu, Jiaxin Duan, Junfei Liu

SPAWNing Structural Priming Predictions from a Cognitively Motivated Parser

Grusha Prasad, Tal Linzen

Jon Chun

On Functional Competence of LLMs for Linguistic Disambiguation

Raihan Kibria, Sheikh Intiser Uddin Dipta, Muhammad Abdullah Adnan

Causal ATE Mitigates Unintended Bias in Controlled Text Generation

Rahul Madhavan, Kahini Wadhawan

Information Association for Language Model Updating by Mitigating LM-Logical Discrepancy

Pengfei Yu, Heng Ji

Critical Questions Generation: Motivation and Challenges

Blanca Calvo Figueras, Rodrigo Agerri

An Empirical Comparison of Vocabulary Expansion and Initialization Approaches For Language Models

Nandini Mundra, Aditya Nanda Kishore Khandavally, Raj Dabre, Ratish Puduppully, Anoop Kunchukuttan

EditEval: An Instruction-Based Benchmark for Text Improvements

Jane Dwivedi-Yu, Timo Schick, Zhengbao Jiang, Maria Lomeli, Patrick Lewis

Transformer verbatim in-context retrieval across time and scale

Kristijan Armeni, Marko Pranjić, Senja Pollak

Global-Pruner: A Stable and Efficient Pruner for Retraining-Free Pruning of Encoder-Based Language Models

Guangzhen Yao, Yuehan Wang, Hui Xu, Long Zhang, Miao Qi

Lossy Context Surprisal Predicts Task-Dependent Patterns in Relative Clause Processing

Kate McCurdy, Michael Hahn

Multi-Cultural Norm Base: Frame-based Norm Discovery in Multi-Cultural Settings

Viet Thanh Pham, Shilin Qu, Farhad Moghimifar, Suraj Sharma, Yuan-Fang Li

Investigating large language models for their competence in extracting grammatically sound sentences from transcribed noisy utterances

Alina Wróblewska

Words That Stick: Using Keyword Cohesion to Improve Text Segmentation

Amit Maraj, Miguel Vargas Martin, Masoud Makrehchi