A Fast Learning Algorithm for Deep Belief Nets

  title={A Fast Learning Algorithm for Deep Belief Nets},
  author={Geoffrey E. Hinton and Simon Osindero and Y. Teh},
  journal={Neural Computation},
  • Geoffrey E. Hinton, Simon Osindero, Y. Teh
  • Published 2006
  • Mathematics, Medicine, Computer Science
  • Neural Computation
  • We show how to use complementary priors to eliminate the explaining-away effects that make inference difficult in densely connected belief nets that have many hidden layers. Using complementary priors, we derive a fast, greedy algorithm that can learn deep, directed belief networks one layer at a time, provided the top two layers form an undirected associative memory. The fast, greedy algorithm is used to initialize a slower learning procedure that fine-tunes the weights using a contrastive… CONTINUE READING
    11,252 Citations
    Sparse Deep Belief Net for Handwritten Digits Classification
    • 1
    A Novel Sparse Deep Belief Network for Unsupervised Feature Learning
    Efficient Learning of Deep Boltzmann Machines
    • 303
    • Highly Influenced
    • PDF
    Exploring Strategies for Training Deep Neural Networks
    • 901
    • PDF
    An Efficient Learning Procedure for Deep Boltzmann Machines
    • 350
    • PDF
    On the quantitative analysis of deep belief networks
    • 407
    • Highly Influenced
    • PDF
    Unsupervised feature learning using Markov deep belief network
    • 6
    Partitioning Large Scale Deep Belief Networks Using Dropout
    Modular deep belief networks that do not forget
    • 11
    • PDF
    Greedy Layer-Wise Training of Deep Networks
    • 2,523
    • Highly Influenced
    • PDF


    Visual Recognition and Inference Using Dynamic Overcomplete Sparse Learning
    • 34
    • PDF
    Connectionist Learning of Belief Networks
    • R. Neal
    • Computer Science
    • Artif. Intell.
    • 1992
    • 562
    • PDF
    On Contrastive Divergence Learning
    • 637
    • PDF
    Knowledge Transfer in Deep convolutional Neural Nets
    • 34
    • PDF
    Optimal unsupervised learning in a single-layer linear feedforward neural network
    • 1,482
    • PDF
    Boosting a weak learning algorithm by majority
    • 973
    Energy-Based Models for Sparse Overcomplete Representations
    • 167
    • PDF