Corpus ID: 420069

Expressive Power and Approximation Errors of Restricted Boltzmann Machines

@inproceedings{Montfar2011ExpressivePA,
  title={Expressive Power and Approximation Errors of Restricted Boltzmann Machines},
  author={Guido Mont{\'u}far and J. Rauh and N. Ay},
  booktitle={NIPS},
  year={2011}
}
  • Guido Montúfar, J. Rauh, N. Ay
  • Published in NIPS 2011
  • Mathematics, Computer Science
  • We present explicit classes of probability distributions that can be learned by Restricted Boltzmann Machines (RBMs) depending on the number of units that they contain, and which are representative for the expressive power of the model. We use this to show that the maximal Kullback-Leibler divergence to the RBM model with n visible and m hidden units is bounded from above by (n – 1) – log(m + 1). In this way we can specify the number of hidden units that guarantees a sufficiently rich model… CONTINUE READING
    On the Representational Efficiency of Restricted Boltzmann Machines
    41
    Geometry and expressive power of conditional restricted Boltzmann machines
    18
    Expressive Power of Conditional Restricted Boltzmann Machines
    4
    Discrete restricted Boltzmann machines
    22
    Universal Approximation Depth and Errors of Narrow Belief Networks with Discrete Units
    30
    Deep Narrow Boltzmann Machines are Universal Approximators
    8
    Kernels and Submodels of Deep Belief Networks
    2
    On the magnitude of parameters of RBMs being universal approximators
    Restricted Boltzmann Machines: Introduction and Review
    14

    References

    Publications referenced by this paper.
    SHOWING 1-10 OF 48 REFERENCES
    Refinements of Universal Approximation Results for Deep Belief Networks and Restricted Boltzmann Machines
    75
    Representational Power of Restricted Boltzmann Machines and Deep Belief Networks
    539
    Mixture Models and Representational Power of RBM ’ s , DBN ’ s and DBM ’ s
    3
    Geometry of the Restricted Boltzmann Machine
    55
    On Contrastive Divergence Learning
    619
    Greedy Layer-Wise Training of Deep Networks
    1607
    Deep Belief Networks Are Compact Universal Approximators
    122
    Greedy Layer-Wise Training of Deep Networks
    2436