Blind Nonnegative Source Separation Using Biological Neural Networks

@article{Pehlevan2017BlindNS,
  title={Blind Nonnegative Source Separation Using Biological Neural Networks},
  author={Cengiz Pehlevan and Sreyas Mohan and Dmitri B. Chklovskii},
  journal={Neural Computation},
  year={2017},
  volume={29},
  pages={2925-2954}
}
Blind source separation—the extraction of independent sources from a mixture—is an important problem for both artificial and natural signal processing. Here, we address a special case of this problem when sources (but not the mixing matrix) are known to be nonnegative—for example, due to the physical nature of the sources. We search for the solution to this problem that can be implemented using biologically plausible neural networks. Specifically, we consider the online setting where the data… 
Blind Bounded Source Separation Using Neural Networks with Local Learning Rules
  • A. Erdogan, C. Pehlevan
  • Computer Science
    ICASSP 2020 - 2020 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)
  • 2020
TLDR
A principled derivation of an adaptive BSM algorithm leads to a recurrent neural network with a clipping nonlinearity that adapts by local learning rules, satisfying an important constraint for both biological plausibility and implementability in neuromorphic hardware.
Bio-NICA: A biologically inspired single-layer network for Nonnegative Independent Component Analysis
TLDR
This work derives a single-layer neural network implementation of NICA satisfying the following 3 constraints, which are relevant for biological systems and the design of neuromorphic hardware: (i) the network operates in the online setting, (ii) the synaptic learning rules are local, and (iii) the neural outputs are nonnegative.
Biologically plausible single-layer networks for nonnegative independent component analysis
TLDR
This work improves upon the work by Pehlevan et al. by deriving 2 algorithms for NICA, each with a biologically plausible single-layer network implementation, each of which maps onto a network with indirect lateral connections mediated by interneurons.
Single Channel Blind Source Separation Under Deep Recurrent Neural Network
TLDR
A new method was given for blind signal processing with artificial intelligence by improving the traditional recurrent neural network structure, and proposing the three layers deep recurrent neural networks to realize single channel blind source separation of nonlinear mixed signals.
A Normative and Biologically Plausible Algorithm for Independent Component Analysis
TLDR
A novel objective function for ICA is proposed from which a biologically plausible NN is derived, including both the neural architecture and the synaptic learning rules, Interestingly, the algorithm relies on modulating synaptic plasticity by the total activity of the output neurons.
Neuroscience-inspired online unsupervised learning algorithms
TLDR
This work developed a family of biologically plausible artificial neural networks (NNs) for unsupervised learning based on optimizing principled objective functions containing a term that matches the pairwise similarity of outputs to the similarity of inputs, hence the name - similarity-based.
Online Bounded Component Analysis: A Simple Recurrent Neural Network with Local Update Rule for Unsupervised Separation of Dependent and Independent Sources
TLDR
A low complexity recurrent neural network structure is proposed for unsupervised separation of both independent and dependent sources from their linear mixtures, generated based on Bounded Component Analysis (BCA) approach.
On the Achievability of Blind Source Separation for High-Dimensional Nonlinear Source Mixtures
TLDR
The proposed theorem theoretically guarantees that applying linear PCA to the inputs can reliably extract a subspace spanned by the linear projections from every hidden source as the major components—and thus projecting the inputs onto their major eigenspace can effectively recover a linear transformation of the hidden sources.
A Spiking Neural Network with Local Learning Rules Derived from Nonnegative Similarity Matching
  • C. Pehlevan
  • Computer Science
    ICASSP 2019 - 2019 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)
  • 2019
TLDR
This work provides a principled derivation of a spiking algorithm for unsupervised learning, starting from the nonnegative similarity matching cost function, and shows in simulations that the algorithm can perform sparse feature extraction and manifold learning.
Bigradient neural network-based quantum particle swarm optimization for blind source separation
TLDR
This paper presents a hybrid method to optimized the ICA method by using the quantum particle swarm optimization method (QPSO) to optimize the Bigradient neural network method that applies to separate mixed signals and recover sources signals.
...
...

References

SHOWING 1-10 OF 60 REFERENCES
Cultured Cortical Neurons Can Perform Blind Source Separation According to the Free-Energy Principle
TLDR
It is discovered that neurons in cultures of dissociated rat cortical cells could learn to represent particular sources while filtering out other signals, and this is the first in vitro (as opposed to in silico) demonstration of neural networks performing blind source separation, and the first formal demonstration of neuronal self-organization under the free energy principle.
A New Learning Algorithm for Blind Signal Separation
TLDR
A new on-line learning algorithm which minimizes a statistical dependency among outputs is derived for blind separation of mixed signals and has an equivariant property and is easily implemented on a neural network like model.
Sparse Representations for the Cocktail Party Problem
TLDR
Here it is provided an example of how sparse overcomplete linear representations can directly solve difficult acoustic signal processing problems, using as an example monaural source separation using solely the cues provided by the differential filtering imposed on a source by its path from its origin to the cochlea.
Blind Separation of Positive Sources by Globally Convergent Gradient Search
TLDR
This letter considers the task of independent component analysis when the independent sources are known to be nonnegative and well grounded, which means that they have a nonzero pdf in the region of zero, and proposes a cost function whose minimum coincides with nonnegativity and derives the gradient algorithm under the whitening constraint.
A Local Learning Rule for Independent Component Analysis
TLDR
This work proposes a new learning rule that is both easy to implement and reliable and outperforms other local learning rules over a wide range of parameters, and can separate independent sources without any preprocessing even if the number of sources is unknown.
Algorithms for nonnegative independent component analysis
TLDR
It is sufficient to find the orthonormal rotation y=Wz of prewhitened sources z=Vx, which minimizes the mean squared error of the reconstruction of z from the rectified version y/sup +/ of y, which shows in particular the fast convergence of the rotation and geodesic methods.
An Information-Maximization Approach to Blind Separation and Blind Deconvolution
TLDR
It is suggested that information maximization provides a unifying framework for problems in "blind" signal processing and dependencies of information transfer on time delays are derived.
ADAPTIVE LATERAL INHIBITION FOR NON-NEGATIVE ICA
TLDR
Under certain reasonable conditions, it appears to be sufficient simply to ensure that the output matrix has diagonal covariance (in addition to the non-negativity constraints) to find the independent basis of the decomposing matrix.
Conditions for nonnegative independent component analysis
TLDR
This work considers the noiseless linear independent component analysis problem, in the case where the hidden sources s are nonnegative, and shows that y is a permutation of the s if and only if y is nonnegative with probability 1.
...
...