Unsupervised Learning

  title={Unsupervised Learning},
  author={Geoffrey E. Hinton and Terrence J. Sejnowski},
  booktitle={Encyclopedia of Machine Learning and Data Mining},
Unsupervised learning studies how systems can learn to represent particular input patterns in a way that reflects the statistical structure of the overall collection of input patterns. By contrast with SUPERVISED LEARNING or REINFORCEMENT LEARNING, there are no explicit target outputs or environmental evaluations associated with each input; rather the unsupervised learner brings to bear prior biases as to what aspects of the structure of the input should be captured in the output. 
Generative models for discovering sparse distributed representations.
  • G. Hinton, Zoubin Ghahramani
  • Computer Science
    Philosophical transactions of the Royal Society of London. Series B, Biological sciences
  • 1997
A hierarchical, generative model that can be viewed as a nonlinear generalization of factor analysis and can be implemented in a neural network that learns to extract sparse, distributed, hierarchical representations is described.
Supervising an Unsupervised Neural Network
A simple plasticity neural network model is proposed that has the ability of storing information as well as storing the association between a pair of inputs and two simple unsupervised learning rules are introduced and a framework to supervise the neural network is introduced.
Unsupervised Learning of Visual Structure
A single-layer network is described that combines fast imprinting in the feedforward stream with lateral interactions to achieve single-epoch unsupervised acquisition of spatially localized features that can support systematic treatment of structured objects.
Unsupervised Learning via Meta-Learning
This work develops an unsupervised meta-learning method that explicitly optimizes for the ability to learn a variety of tasks from small amounts of data, and acquires a learning algorithm without any labeled data that is applicable to a wide range of downstream classification tasks.
Boosting Reinforcement Learning with Unsupervised Feature Extraction
While pretrained filters improve object detection tasks, the influence of convolutional filters that were pretrained on a supervised classification task, a Convolutional Autoencoder and Slow Feature Analysis are investigated in an end-to-end architecture.
Finding Clusters and Components by Unsupervised Learning
  • E. Oja
  • Computer Science
  • 2004
A tutorial survey on some recent approaches to unsupervised machine learning in the context of statistical pattern recognition, with examples such as linear and nonlinear independent component analysis and topological maps.
Unsupervised learning in neural computation
  • E. Oja
  • Computer Science
    Theor. Comput. Sci.
  • 2002
"Memory foam" approach to unsupervised learning
An alternative approach to construct an artificial learning system, which naturally learns in an unsupervised manner, which automatically shapes its vector field in response to the input signal.
The Rôle of a priori Biases in Unsupervised Learning of Visual Representations : A Robotics Experiment
This paper uses an autonomous robot whose learning is focused on “relevant” stimuli to study the effects of a priori biases on unsupervised learning and shows how the exploitation of temporal continuity allows the robot to generalize its innate knowledge of what stimuli are relevant to new contexts.


The Helmholtz Machine
A way of finessing this combinatorial explosion by maximizing an easily computed lower bound on the probability of the observations is described, viewed as a form of hierarchical self-supervised learning that may relate to the function of bottom-up and top-down cortical processing pathways.
Connectionist Learning Procedures
Unsupervised Learning: Foundations of Neural Computation
The edited volume provides a sample of important works on unsupervised learning, which cut across the fields of neural networks, and some of the most influential titles of late.
Learning and relearning in Boltzmann machines
This chapter contains sections titled: Relaxation Searches, Easy and Hard Learning, The Boltzmann Machine Learning Algorithm, An Example of Hard Learning, Achieving Reliable Computation with
A theory for cerebral neocortex
  • D. Marr
  • Biology
    Proceedings of the Royal Society of London. Series B. Biological Sciences
  • 1970
It is shown how a climbing fibre input to the correct cell can cause that cell to perform a mountain-climbing operation in an underlying probability space, that will lead it to respond to a class of events for which it is appropriate to code.
Self-organization in a perceptual network
It is shown that even a single developing cell of a layered network exhibits a remarkable set of optimization properties that are closely related to issues in statistics, theoretical physics, adaptive signal processing, the formation of knowledge representation in artificial intelligence, and information theory.