CoNLL 2023

72 papers

Optimizing GPT-2 Pretraining on BabyLM Corpus with Difficulty-based Sentence Reordering

Nasim Borazjanizadeh

On the effect of curriculum learning with developmental data for grammar acquisition

Mattia Opper, J. Morrison, N. Siddharth

Pre-training LLMs using human-like development data corpus

Khushi Bhardwaj, Raj Sanjay Shah, Sashank Varma

Increasing The Performance of Cognitively Inspired Data-Efficient Language Models via Implicit Structure Building

Omar Momen, David Arps, Laura Kallmeyer

Towards more Human-like Language Models based on Contextualizer Pretraining Strategy

Chenghao Xiao, G Thomas Hudson, Noura Al Moubayed

Lil-Bevo: Explorations of Strategies for Training Language Models in More Humanlike Ways

Venkata S Govindarajan, Juan Diego Rodriguez, Kaj Bostrom, Kyle Mahowald

Better Together: Jointly Using Masked Latent Semantic Modeling and Masked Language Modeling for Sample Efficient Pre-training

Gábor Berend

BabyLM Challenge: Curriculum learning based on sentence complexity approximating language acquisition

Miyu Oba, Akari Haga, Akiyo Fukatsu, Yohei Oseki

Baby Llama: knowledge distillation from an ensemble of teachers trained on a small dataset with no performance penalty

Inar Timiryasov, Jean-Loup Tastet

Mmi01 at The BabyLM Challenge: Linguistically Motivated Curriculum Learning for Pretraining in Low-Resource Settings

Maggie Mi

A surprisal oracle for active curriculum language modeling

Xudong Hong, Sharid Loáiciga, Asad Sayeed

WhisBERT: Multimodal Text-Audio Language Modeling on 100M Words

Lukas Wolf, Klemen Kotar, Greta Tuckute, Eghbal Hosseini, Tamar I. Regev

Not all layers are equally as important: Every Layer Counts BERT

Lucas Georges Gabriel Charpentier, David Samuel

Mean BERTs make erratic language teachers: the effectiveness of latent bootstrapping in low-resource settings

David Samuel

McGill BabyLM Shared Task Submission: The Effects of Data Formatting and Structural Biases

Ziling Cheng, Rahul Aralikatte, Ian Porada, Cesare Spinoso-Di Piano, Jackie CK Cheung

Byte-ranked Curriculum Learning for BabyLM Strict-small Shared Task 2023

Justin DeBenedetto

BabyStories: Can Reinforcement Learning Teach Baby Language Models to Write Better Stories?

Xingmeng Zhao, Tongnian Wang, Sheri Osborn, Anthony Rios

CogMemLM: Human-Like Memory Mechanisms Improve Performance and Cognitive Plausibility of LLMs

Lukas Thoma, Ivonne Weyers, Erion Çano, Stefan Schweter, Jutta L Mueller

ToddlerBERTa: Exploiting BabyBERTa for Grammar Learning and Language Understanding

Omer Veysel Cagatan

Baby’s CoThought: Leveraging Large Language Models for Enhanced Reasoning in Compact Models

Zheyu Zhang, Han Yang, Bolei Ma, David Rügamer, Ercong Nie

Large GPT-like Models are Bad Babies: A Closer Look at the Relationship between Linguistic Competence and Psycholinguistic Measures

Julius Steuer, Marius Mosbach, Dietrich Klakow

Acquiring Linguistic Knowledge from Multimodal Input

Theodor Amariucai, Alexander Scott Warstadt

CLIMB – Curriculum Learning for Infant-inspired Model Building

Richard Diehl Martinez, Zébulon Goriely, Hope McGovern, Christopher Davis, Andrew Caines

Can training neural language models on a curriculum with developmentally plausible data improve alignment with human reading behavior?

Aryaman Chobey, Oliver Smith, Anzi Wang, Grusha Prasad

Too Much Information: Keeping Training Simple for BabyLMs

Lukas Edman, Lisa Bylinina

Penn & BGU BabyBERTa+ for Strict-Small BabyLM Challenge

Yahan Yang, Elior Sulem, Insup Lee, Dan Roth

ChapGTP, ILLC’s Attempt at Raising a BabyLM: Improving Data Efficiency by Automatic Task Formation

Jaap Jumelet, Michael Hanna, Marianne de Heer Kloots, Anna Langedijk, Charlotte Pouw

Grammar induction pretraining for language modeling in low resource contexts

Xuanda Chen, Eva Portelance

Mini Minds: Exploring Bebeshka and Zlata Baby Models

Irina Proskurina, Guillaume Metzler, Julien Velcin

Tiny Language Models Enriched with Multimodal Knowledge from Multiplex Networks

Clayton Fields, Osama Natouf, Andrew McMains, Catherine Henry, Casey Kennington

GPT-wee: How Small Can a Small Language Model Really Get?

Bastian Bunzeck, Sina Zarrieß

Findings of the BabyLM Challenge: Sample-Efficient Pretraining on Developmentally Plausible Corpora

Alex Warstadt, Aaron Mueller, Leshem Choshen, Ethan Wilcox, Chengxu Zhuang

REFER: An End-to-end Rationale Extraction Framework for Explanation Regularization

Mohammad Reza Ghasemi Madani, Pasquale Minervini

Implications of Annotation Artifacts in Edge Probing Test Datasets

Sagnik Ray Choudhury, Jushaan Kalra

Cross-Document Event Coreference Resolution: Instruct Humans or Instruct GPT?

Jin Zhao, Nianwen Xue, Bonan Min

Future Lens: Anticipating Subsequent Tokens from a Single Hidden State

Koyena Pal, Jiuding Sun, Andrew Yuan, Byron Wallace, David Bau

Tree-shape Uncertainty for Analyzing the Inherent Branching Bias of Unsupervised Parsing Models

Taiga Ishii, Yusuke Miyao

Exploring Transformers as Compact, Data-efficient Language Models

Clayton Fields, Casey Kennington

Strategies to Improve Low-Resource Agglutinative Languages Morphological Inflection

Gulinigeer Abudouwaili, Wayit Ablez, Kahaerjiang Abiderexiti, Aishan Wumaier, Nian Yi

Challenging the “One Single Vector per Token” Assumption

Mathieu Dehouck

Alignment via Mutual Information

Shinjini Ghosh, Yoon Kim, Ramon Fernandez Astudillo, Tahira Naseem, Jacob Andreas

PSST! Prosodic Speech Segmentation with Transformers

Nathan Roll, Calbert Graham, Simon Todd

The Impact of Familiarity on Naming Variation: A Study on Object Naming in Mandarin Chinese

Yunke He, Xixian Liao, Jialing Liang, Gemma Boleda

MuLER: Detailed and Scalable Reference-based Evaluation

Taelin Karidi, Leshem Choshen, Gal Patel, Omri Abend

JaSPICE: Automatic Evaluation Metric Using Predicate-Argument Structures for Image Captioning Models

Yuiga Wada, Kanta Kaneda, Komei Sugiura

How Fragile is Relation Extraction under Entity Replacements?

Yiwei Wang, Bryan Hooi, Fei Wang, Yujun Cai, Yuxuan Liang

A Block Metropolis-Hastings Sampler for Controllable Energy-based Text Generation

Jarad Forristal, FatemehSadat Mireshghallah, Greg Durrett, Taylor Berg-Kirkpatrick

Theory of Mind in Large Language Models: Examining Performance of 11 State-of-the-Art models vs. Children Aged 7-10 on Advanced Tests

Max van Duijn, Bram van Dijk, Tom Kouwenhoven, Werner de Valk, Marco Spruit

HNC: Leveraging Hard Negative Captions towards Models with Fine-Grained Visual-Linguistic Comprehension Capabilities

Esra Dönmez, Pascal Tilli, Hsiu-Yu Yang, Ngoc Thang Vu, Carina Silberer

ChiSCor: A Corpus of Freely-Told Fantasy Stories by Dutch Children for Computational Linguistics and Cognitive Science

Bram van Dijk, Max van Duijn, Suzan Verberne, Marco Spruit

Revising with a Backward Glance: Regressions and Skips during Reading as Cognitive Signals for Revision Policies in Incremental Processing

Brielen Madureira, Pelin Çelikkol, David Schlangen

Med-HALT: Medical Domain Hallucination Test for Large Language Models

Ankit Pal, Logesh Kumar Umapathi, Malaikannan Sankarasubbu

Mind the instructions: a holistic evaluation of consistency and interactions in prompt-based learning

Lucas Weber, Elia Bruni, Dieuwke Hupkes

The Validity of Evaluation Results: Assessing Concurrence Across Compositionality Benchmarks

Kaiser Sun, Adina Williams, Dieuwke Hupkes

Attribution and Alignment: Effects of Local Context Repetition on Utterance Production and Comprehension in Dialogue

Aron Molnar, Jaap Jumelet, Mario Giulianelli, Arabella Sinclair

Syntactic Inductive Bias in Transformer Language Models: Especially Helpful for Low-Resource Languages?

Luke Gessler, Nathan Schneider

Towards Better Evaluation of Instruction-Following: A Case-Study in Summarization

Ondrej Skopek, Rahul Aralikatte, Sian Gooding, Victor Cărbune

Enhancing Code-mixed Text Generation Using Synthetic Data Filtering in Neural Machine Translation

Dama Sravani, Radhika Mamidi

Quantifying Information of Tokens for Simple and Flexible Simultaneous Machine Translation

Donghyun Lee, Minkyung Park, Byung-Jun Lee

Quirk or Palmer: A Comparative Study of Modal Verb Frameworks with Annotated Datasets

Risako Owan, Maria Gini, Dongyeop Kang

On the utility of enhancing BERT syntactic bias with Token Reordering Pretraining

Yassir El Mesbahi, Atif Mahmud, Abbas Ghaddar, Mehdi Rezagholizadeh, Phillippe Langlais

Structural Ambiguity and its Disambiguation in Language Model Based Parsers: the Case of Dutch Clause Relativization

Gijs Wijnholds, Michael Moortgat

A Minimal Approach for Natural Language Action Space in Text-based Games

Dongwon Ryu, Meng Fang, Gholamreza Haffari, Shirui Pan, Ehsan Shareghi

PROPRES: Investigating the Projectivity of Presupposition with Various Triggers and Environments

Daiki Asami, Saku Sugawara

A Comparative Study on Textual Saliency of Styles from Eye Tracking, Annotations, and Language Models

Karin de Langis, Dongyeop Kang

ArchBERT: Bi-Modal Understanding of Neural Architectures and Natural Languages

Mohammad Akbari, Saeed Ranjbar Alvar, Behnam Kamranian, Amin Banitalebi-Dehkordi, Yong Zhang

Investigating the Nature of Disagreements on Mid-Scale Ratings: A Case Study on the Abstractness-Concreteness Continuum

Urban Knupleš, Diego Frassinelli, Sabine Schulte im Walde

Humans and language models diverge when predicting repeating text

Aditya Vaidya, Javier Turek, Alexander Huth

On the Effects of Structural Modeling for Neural Semantic Parsing

Xiang Zhang, Shizhu He, Kang Liu, Jun Zhao

The Zipfian Challenge: Learning the statistical fingerprint of natural languages

Christian Bentz

ToMChallenges: A Principle-Guided Dataset and Diverse Evaluation Tasks for Exploring Theory of Mind

Xiaomeng Ma, Lingyu Gao, Qihui Xu

Can Language Models Be Tricked by Language Illusions? Easier with Syntax, Harder with Semantics

Yuhan Zhang, Edward Gibson, Forrest Davis