← Back to papers
2023 ICML ICML 2023

Quantitative Universal Approximation Bounds for Deep Belief Networks

Abstract

We show that deep belief networks with binary hidden units can approximate any multivariate probability density under very mild integrability requirements on the parental density of the visible nodes. The approximation is measured in the $L^q$-norm for $q\in[1,\infty]$ ($q=\infty$ corresponding to the supremum norm) and in Kullback-Leibler divergence. Furthermore, we establish sharp quantitative bounds on the approximation error in terms of the number of hidden units.

🌉 Interdisciplinary Bridge — Artificial Intelligence and Deep Learning and Machine Learning
🐝 Cross-Pollinator — Artificial Intelligence, Computer Science, Computer Vision, Data Science & Analytics, Deep Learning, Healthcare & Medicine, Interdisciplinary, Knowledge & Reasoning, Machine Learning, Mathematics & Optimization, Natural Language Processing, Reinforcement Learning, Robotics, Security & Privacy, Speech & Audio

Authors

Julian Sieber , Johann Gehringer

Topics

Artificial Intelligence > Bayesian & Probabilistic > Probabilistic Modeling Machine Learning > Optimization & Theory > Learning Theory Machine Learning > Optimization & Theory > Theory Deep Learning > Architectures > Neural Networks

Keywords

learning theory universal approximation kullback-leibler divergence probability density probabilistic model deep belief network approximation bound neural network
Download PDF

Related papers

Orthogonality-Enforced Latent Space in Autoencoders: An Approach to Learning Disentangled Representations 2023
On the Convergence Rate of Gaussianization with Random Rotations 2023
Synthetic Prompting: Generating Chain-of-Thought Demonstrations for Large Language Models 2023
Generative Adversarial Symmetry Discovery 2023
Function-Space Regularization in Neural Networks: A Probabilistic Perspective 2023