A multi-layer sparse coding network learns contour coding from natural images
We consider the noiseless linear independent component analysis problem, in the case where the hidden sources s are nonnegative. We assume that the random variables si are well grounded in that they have a nonvanishing probability density function (PDF) in the (positive) neighborhood of zero. For an orthonormal rotation y=Wx of prewhitened observations x=QAs, under certain reasonable conditions we show that y is a permutation of the s (apart from a scaling factor) if and only if y is nonnegative with probability 1. We suggest that this may enable the construction of practical learning algorithms, particularly for sparse nonnegative sources.