Sparse coding of sensory inputs

@article{Olshausen2004SparseCO,
  title={Sparse coding of sensory inputs},
  author={Bruno A. Olshausen and David J. Field},
  journal={Current Opinion in Neurobiology},
  year={2004},
  volume={14},
  pages={481-487}
}
Several theoretical, computational, and experimental studies suggest that neurons encode sensory information using a small number of active neurons at any given point in time. This strategy, referred to as 'sparse coding', could possibly confer several advantages. First, it allows for increased storage capacity in associative memories; second, it makes the structure in natural signals explicit; third, it represents complex data in a way that is easier to read out at subsequent levels of… Expand
2 Sparse Coding in the Neocortex
Sparse coding has been proposed as a guiding principle in neural representations of sensory input, particularly in the visual system. Because sparse codes are defined as representations with lowExpand
Sparse Coding in the Neocortex
TLDR
It is argued that larger brains have evolved strategies that allow neurons to be both highly selective and invariant across irrelevant dimensions, and this invariance is what allows such highly sparse codes to be useful. Expand
A Simple Connectivity Scheme for Sparse Coding in an Olfactory System
TLDR
Analysis of the statistics of connectivity in an olfactory network that contributes to the generation of sparse codes indicates that such connectivity indeed maximizes the difference between input vectors to KCs and helps to explain their high specificity. Expand
Questioning the role of sparse coding in the brain
TLDR
Combining silent synapses and inhibitory interneurons can permit learning speed and memory capacity that was previously ascribed to sparse coding only and improves the capacity for generalization and facilitates learning of models of a complex and high-dimensional reality. Expand
Synaptic Learning Rules and Sparse Coding in a Model Sensory System
TLDR
Activity-dependent plasticity drove the observed specificity, reliability, and expected persistence of odor representations, suggesting a role for plasticity in information processing and making a testable prediction about synaptic plasticity at AL-MB synapses. Expand
Cortical connectivity and sensory coding
TLDR
Research is beginning to reveal how the connectivity of individual neurons relates to the sensory features they encode, how differences in the connectivity patterns of different cortical cell classes enable them to encode information using different strategies, and how feedback connections from higher-order cortex allow sensory information to be integrated with behavioural context. Expand
Neurally plausible sparse coding via local competition and thresholding
While evidence indicates that neural systems may be employing sparse approximations to represent sensed stimuli, the mechanisms underlying this ability are not known. We present a class of neurallyExpand
Efficient computation via sparse coding in electrosensory neural networks
TLDR
A systems level understanding arises via linkage between cellular differentiation and network architecture, revealed by in vitro and in vivo analyses, while computational modeling reveals how single cell dynamics and connectivity shape the sparsification process. Expand
Brain at work: time, sparseness and superposition principles.
TLDR
It is shown that synchronization of spikes is a fast and efficient mode to encode the representation of objects based on feature bindings and presents an interesting venue in probing brain encoding mechanisms as it allows the functional establishment of multi-layered and time-conditioned neuronal networks or multislice networks. Expand
Efficient transformation of an auditory population code in a small sensory system
TLDR
It is concluded that optimal coding principles do apply to the early auditory system of the grasshopper, despite its size constraints, and the inputs are not encoded in a systematic, map-like fashion as in many larger sensory systems. Expand
...
1
2
3
4
5
...

References

SHOWING 1-10 OF 70 REFERENCES
Sparse coding in the primate cortex
TLDR
There are now strong theoretical reasons and experimental evidence suggesting that the brain adopts a compromise between these extremes which is often referred to as sparse coding. Expand
What Is the Goal of Sensory Coding?
  • D. Field
  • Mathematics, Computer Science
  • Neural Computation
  • 1994
TLDR
It is proposed that compact coding schemes are insufficient to account for the receptive field properties of cells in the mammalian visual pathway and suggested that natural scenes, to a first approximation, can be considered as a sum of self-similar local functions (the inverse of a wavelet). Expand
Natural image statistics and neural representation.
TLDR
It has long been assumed that sensory neurons are adapted to the statistical properties of the signals to which they are exposed, but recent developments in statistical modeling have enabled researchers to study more sophisticated statistical models for visual images, to validate these models empirically against large sets of data, and to begin experimentally testing the efficient coding hypothesis. Expand
The Cost of Cortical Computation
  • P. Lennie
  • Medicine, Biology
  • Current Biology
  • 2003
TLDR
For human cortex, the cost of individual spikes is estimated, then, from the known energy consumption of cortex, how many neurons can be active concurrently is established. Expand
Sparse coding and decorrelation in primary visual cortex during natural vision.
Theoretical studies suggest that primary visual cortex (area V1) uses a sparse code to efficiently represent natural scenes. This issue was investigated by recording from V1 neurons in awake behavingExpand
Internal representations for associative memory
TLDR
A class of feed forward neural network models for associative content addressable memory (ACAM) which utilize sparse internal representations for stored data and study two classes of internal label representations: the unary representation and various sparse, distributed representations. Expand
A theory of maximizing sensory information
  • J. H. Hateren
  • Mathematics, Computer Science
  • Biological Cybernetics
  • 2004
A theory is developed on the assumption that early sensory processing aims at maximizing the information rate in the channels connecting the sensory system to more central parts of the brain, whereExpand
Sparse population coding of faces in the inferotemporal cortex.
TLDR
Population analysis of cells in the temporal cortex of monkeys showed that information is carried at the level of the population and that representations of complex stimuli in the higher visual areas may take the form of a sparse population code. Expand
Sparse coding with an overcomplete basis set: A strategy employed by V1?
TLDR
These deviations from linearity provide a potential explanation for the weak forms of non-linearity observed in the response properties of cortical simple cells, and they further make predictions about the expected interactions among units in response to naturalistic stimuli. Expand
From synchrony to sparseness
TLDR
The neurons in the antennal lobe of the locust are shown to encode the identity of odorants using spatially distributed synchronized patterns of neural activity, which is transformed from a dense spatial-temporal code into a sparse code. Expand
...
1
2
3
4
5
...