Information Theory

Information Theory and Linguistic Variation: A Study of Brazilian and European Portuguese COLING 2025 Language Models Resist Alignment: Evidence From Data Compression ACL 2025 Learning Interpretable Queries for Explainable Image Classification with Information Pursuit ICCV 2025 Mapping 1,000+ Language Models via the Log-Likelihood Vector ACL 2025 Partial Information Decomposition for Data Interpretability and Feature Selection AISTATS 2025 Pre-trained Models Perform the Best When Token Distributions Follow Zipf’s Law EMNLP 2025 Quantifying Knowledge Distillation using Partial Information Decomposition AISTATS 2025 Quantifying Semantic Emergence in Language Models ACL 2025 Simple Yet Effective: An Information-Theoretic Approach to Multi-LLM Uncertainty Quantification EMNLP 2025 Speculative Sampling via Exponential Races ACL 2025 Stealthy Shield Defense: A Conditional Mutual Information-Based Approach against Black-Box Model Inversion Attacks ICLR 2025 Structured IB: Improving Information Bottleneck with Structured Feature Learning AAAI 2025 SUMI-IFL: An Information-Theoretic Framework for Image Forgery Localization with Sufficiency and Minimality Constraints AAAI 2025 Systematic Generalization in Language Models Scales with Information Entropy ACL 2025 The Causal Information Bottleneck and Optimal Causal Variable Abstractions UAI 2025 The Exception of Humor: Iconicity, Phonemic Surprisal, Memory Recall, and Emotional Associations COLING 2025 The Inverse Scaling Effect of Pre-Trained Language Model Surprisal Is Not Due to Data Leakage ACL 2025 Theoretical Insights in Model Inversion Robustness and Conditional Entropy Maximization for Collaborative Inference Systems CVPR 2025 The Tug of War Within: Mitigating the Fairness-Privacy Conflicts in Large Language Models ACL 2025 West Germanic noun-noun compounds and the morphology-syntax trade-off NAACL 2025 What Makes for Good Image Captions? EMNLP 2025 A Benchmark Suite for Evaluating Neural Mutual Information Estimators on Unstructured Datasets NeurIPS 2024 Accelerating Relative Entropy Coding with Space Partitioning NeurIPS 2024 Analytically deriving Partial Information Decomposition for affine systems of stable and convolution-closed distributions NeurIPS 2024 An Information-Theoretic Approach to Analyze NLP Classification Tasks ACL 2024