Self-organization in a perceptual network

@article{Linsker1988SelforganizationIA,
  title={Self-organization in a perceptual network},
  author={Ralph Linsker},
  journal={Computer},
  year={1988},
  volume={21},
  pages={105-117}
}
  • R. Linsker
  • Published 1 March 1988
  • Computer Science
  • Computer
The emergence of a feature-analyzing function from the development rules of simple, multilayered networks is explored. It is shown that even a single developing cell of a layered network exhibits a remarkable set of optimization properties that are closely related to issues in statistics, theoretical physics, adaptive signal processing, the formation of knowledge representation in artificial intelligence, and information theory. The network studied is based on the visual system. These results… 

Figures from this paper

Information maximization in a network of linear neurons

Linsker showed in [7, 8, 9] that the self-organization process leading to these receptive field structures is consistent with the principle of maximum information preservation, or “infomax” principle for short.

Development of feature detectors by self-organization

A two-layered network of linear neurons that organizes itself as to extract the complete information contained in a set of presented patterns, and the output units become detectors of orthogonal features, similar to ones found in the brain of mammals.

Self-organizing maps and adaptive filters

This contribution reports results on two important network models, one of which is a hierarchical network for principal component analysis and the other a self-organizing feature maps based on Kohonen's model.

Parallel hierarchical networks in communications

The multistage approach to image processing, described in this paper, comprises both main types of cortical multistages, which maps input images into a flexible hierarchy which reflects the complexity of the image data.

Towards a Theory of Early Visual Processing

A theory of the early processing in the mammalian visual pathway is proposed and the solution for the class of encodings to which the parvocellular retinal processing belongs, namely linear and nondivergent transformations is found.

Scaling in a Hierarchical Unsupervised Network 1

A novel hierarchical, generative model that can be viewed as a nonlinear generalization of factor analysis and can be implemented in a neural network performs perceptual inference in a probabilistically consistent manner by using top-down, bottom-up and lateral connections.

Scaling in a hierarchical unsupervised network

This work demonstrates that a novel hierarchical, generative model that can be viewed as a nonlinear generalisation of factor analysis and can be implemented in a neural network performs perceptual inference in a probabilistically consistent manner by using top-down, bottom-up and lateral connections.

Emergence of Retinotopic Orientation Map by Hebb-rule: Network Model and Analysis

The emergence of cortical maps is modeled with a neuronal network with stochastic neuronal dynamics and a Hebb-rule for the coupling dynamics. The network is analyzed systematically. As a result, the

Learning spatially coherent properties of the visual world in connectionist networks

Simulations show that using an information-theoretic algorithm called IMAX, a network can be trained to represent depth by observing random dot stereograms of surfaces with continuously varying disparities.

Analytic Solutions to the Formation of Feature-Analysing Cells of a Three-Layer Feedforward Visual Information Processing Neural Net

Analytic solutions to the information-theoretic evolution equation of the connection strength of a three-layer feedforward neural net for visual information processing are presented and a symmetry-breaking mechanism has been identified to be responsible for the changes of the morphology of the receptive field.
...

References

SHOWING 1-10 OF 21 REFERENCES

Towards an Organizing Principle for a Layered Perceptual Network

An information-theoretic optimization principle is proposed for the development of each processing stage of a multilayered perceptual network that maximizes the information that the output signal values convey about the input signals values, subject to certain constraints and in the presence of processing noise.

From basic network principles to neural architecture (series)

This paper shows that, in the presence of lateral connections between developing orientation cells, these cells self-organize into banded patterns of cells of similar orientation, similar to the "orientation columns" found in mammalian visual cortex.

From basic network principles to neural architecture: emergence of spatial-opponent cells.

  • R. Linsker
  • Biology
    Proceedings of the National Academy of Sciences of the United States of America
  • 1986
This paper is the first of three that address the origin and organization of feature-analyzing cells in simple systems governed by biologically plausible development rules, and introduces the theory of "modular self-adaptive networks," of which this system is an example, and explicitly demonstrates the emergence of a layer of spatial-opponent cells.

Neural networks and physical systems with emergent collective computational abilities.

  • J. Hopfield
  • Computer Science
    Proceedings of the National Academy of Sciences of the United States of America
  • 1982
A model of a system having a large number of simple equivalent components, based on aspects of neurobiology but readily adapted to integrated circuits, produces a content-addressable memory which correctly yields an entire memory from any subpart of sufficient size.

Ferrier lecture - Functional architecture of macaque monkey visual cortex

  • D. HubelT. Wiesel
  • Biology
    Proceedings of the Royal Society of London. Series B. Biological Sciences
  • 1977
In most respects the above description fits the newborn monkey just as well as the adult, suggesting that area 17 is largely genetically programmed.

Functional architecture of macaque monkey visual cortex

By four independent anatomical methods it has been shown that these columns have an ocular dominance column all cells respond preferentially to the same eye, in that cells with common physiological properties are grouped together in vertically organized systems of columns.

Self-Organization and Associative Memory

The purpose and nature of Biological Memory, as well as some of the aspects of Memory Aspects, are explained.

Simplified neuron model as a principal component analyzer

  • E. Oja
  • Biology
    Journal of mathematical biology
  • 1982
A simple linear neuron model with constrained Hebbian-type synaptic modification is analyzed and a new class of unconstrained learning rules is derived. It is shown that the model neuron tends to

Brain mechanisms of vision.

From basic network principles to neural architecture