Deep Belief Networks Are Compact Universal Approximators

  title={Deep Belief Networks Are Compact Universal Approximators},
  author={Nicolas Le Roux and Yoshua Bengio},
  journal={Neural Computation},
Deep belief networks (DBN) are generative models with many layers of hidden causal variables, recently introduced by Hinton, Osindero, and Teh (2006), along with a greedy layer-wise unsupervised learning algorithm. Building on Le Roux and Bengio (2008) and Sutskever and Hinton (2008), we show that deep but narrow generative networks do not require more parameters than shallow ones to achieve universal approximation. Exploiting the proof technique, we prove that deep but narrow feedforward… CONTINUE READING
Highly Cited
This paper has 204 citations. REVIEW CITATIONS

From This Paper

Figures, tables, and topics from this paper.
62 Citations
17 References
Similar Papers


Publications citing this paper.
Showing 1-10 of 62 extracted citations

204 Citations

Citations per Year
Semantic Scholar estimates that this publication has 204 citations based on the available data.

See our FAQ for additional information.


Publications referenced by this paper.

Similar Papers

Loading similar papers…