conftrace_

2012 NeurIPS NeurIPS 2012

Meta-Gaussian Information Bottleneck

Abstract

We present a reformulation of the information bottleneck (IB) problem in terms of copula, using the equivalence between mutual information and negative copula entropy. Focusing on the Gaussian copula we extend the analytical IB solution available for the multivariate Gaussian case to distributions with a Gaussian dependence structure but arbitrary marginal densities, also called meta-Gaussian distributions. This opens new possibles applications of IB to continuous data and provides a solution more robust to outliers.

🌉 Interdisciplinary Bridge - Artificial Intelligence and Mathematics & Optimization

🐣 Hot Topic Early Bird - mutual information

Authors

Melanie Rey, Volker Roth

Topics

Artificial Intelligence > Bayesian & Probabilistic > Probabilistic Modeling Machine Learning > Core Methods > Representation Learning Machine Learning > Optimization & Theory > Statistical Learning Deep Learning > Models > Variational Inference Mathematics & Optimization > Mathematics > Information Theory Mathematics & Optimization > Mathematics > Statistics Machine Learning > Bayesian & Probabilistic > Probabilistic Modeling Machine Learning > Optimization & Theory > Information Theory Machine Learning > Optimization & Theory > Statistics

Keywords

probabilistic modeling variational inference information bottleneck mutual information outlier robustness gaussian copula continuous data copula entropy

More from NeurIPS 2012

All NeurIPS 2012 papers →

Small-Variance Asymptotics for Exponential Family Dirichlet Process Mixture Models

Deep Representations and Codes for Image Auto-Annotation

Patient Risk Stratification for Hospital-Associated C. diff as a Time-Series Classification Task

Efficient Monte Carlo Counterfactual Regret Minimization in Games with Many Player Actions

Fusion with Diffusion for Robust Visual Tracking

Convolutional-Recursive Deep Learning for 3D Object Classification