Variable-size Discovery and Representation of Atomic Propositions
Abstract
AbstractSentence embeddings represent meaning as a single fixed-size vector, regardless of the complexity of the input sentence. This design compresses all propositional structure into a single point, losing part of the internal structure of the sentence in the process. To address this, we introduce Atomic Proposition Embeddings (APE) to dynamically discover and encode atomic propositions in sentences as variable-size bags of vectors. We use a slot mechanism, inspired by object detection in computer vision, over an encoder backbone. APE is trained on a proposition-decomposed version of WikiSplit, and matches or outperforms sentence encoders up to 15x its size on PAWS-Wiki and SNLI with no task-specific fine-tuning, using proposition-level scoring rules at inference time. We further introduce PropSearch, a retrieval benchmark for proposition-level embeddings in which models must retrieve source sentences given atomic propositions. APE leads on PropSearch over proposition-level alternatives, with its advantage growing with sentence complexity. Ablations confirm the gains are architectural rather than data-driven and that APE can generalize over its training signal. We release our code, data and models for research purposes at https://github.com/SapienzaNLP/APE.