information theory

489 papers

Explore in graph

Also known as

IP IT

Co-occurring keywords

mutual information (717) representation learning (6174) sample complexity (1158) language model (4573) generalization bound (649) learning theory (518) large language model (12755) causal inference (1619) statistical learning (350) unsupervised learning (3255)

Papers

Information Locality as an Inductive Bias for Neural Language Models ACL 2025

Trade-Offs Between Information and Crowding in Sequential Decisions (Student Abstract) AAAI 2025

Information-Theoretic Generative Clustering of Documents AAAI 2025

Noise May Drown Out Words but Foster Compositionality: The Advantage of the Erasure and Deletion Noisy Channels on Emergent Communication IJCNLP 2025

Confidence-Aware Reasoning: Optimizing Self-Guided Thinking Trajectories in Large Reasoning Models EMNLP 2025

West Germanic noun-noun compounds and the morphology-syntax trade-off NAACL 2025

Collaborative Rational Speech Act: Pragmatic Reasoning for Multi-Turn Dialog EMNLP 2025

An Information-theoretic Multi-task Representation Learning Framework for Natural Language Understanding AAAI 2025

A Grounded Typology of Word Classes NAACL 2025

Fundamental Limits of Prompt Compression: A Rate-Distortion Framework for Black-Box Language Models NIPS 2024

Q-Tuning: Queue-based Prompt Tuning for Lifelong Few-shot Language Learning NAACL 2024

Looks Too Good To Be True: An Information-Theoretic Analysis of Hallucinations in Generative Restoration Models NIPS 2024

Exploring Word Formation Trends in Written, Spoken, Translated and Interpreted European Parliament Data – A Case Study on Initialisms in English and German COLING 2024

Diff-eRank: A Novel Rank-Based Metric for Evaluating Large Language Models NIPS 2024

Increasing information for model predictive control with semi-Markov decision processes L4DC 2024

BVT-IMA: Binary Vision Transformer with Information-Modified Attention AAAI 2024

Analytically deriving Partial Information Decomposition for affine systems of stable and convolution-closed distributions NIPS 2024

The Semantic Relations in LLMs: An Information-theoretic Compression Approach COLING 2024

Information-theoretic hypothesis generation of relative cue weighting for the voicing contrast INTERSPEECH 2024

Qualitative Mechanism Independence NIPS 2024

Online Estimation via Offline Estimation: An Information-Theoretic Framework NIPS 2024

Did You Get It? A Zero-Shot Approach to Locate Information Transfers in Conversations COLING 2024

Semi-Supervised Sparse Gaussian Classification: Provable Benefits of Unlabeled Data NIPS 2024

On Causality in Domain Adaptation and Semi-Supervised Learning: an Information-Theoretic Analysis for Parametric Models JMLR 2024

Information-Theoretic Generalization Bounds for Transductive Learning and its Applications JMLR 2024