• Corpus ID: 2095855

Algorithms for Non-negative Matrix Factorization

@inproceedings{Lee2000AlgorithmsFN,
  title={Algorithms for Non-negative Matrix Factorization},
  author={Daniel D. Lee and H. Sebastian Seung},
  booktitle={NIPS},
  year={2000}
}
Non-negative matrix factorization (NMF) has previously been shown to be a useful decomposition for multivariate data. Two different multiplicative algorithms for NMF are analyzed. They differ only slightly in the multiplicative factor used in the update rules. One algorithm can be shown to minimize the conventional least squares error while the other minimizes the generalized Kullback-Leibler divergence. The monotonic convergence of both algorithms can be proven using an auxiliary function… 

Figures from this paper

Alternating direction method of multipliers for non-negative matrix factorization with the beta-divergence
  • Dennis L. Sun, C. Févotte
  • Computer Science
    2014 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)
  • 2014
TLDR
This work develops an algorithm, based on the alternating direction method of multipliers, that tackles NMF problems whose cost function is a beta-divergence, a broad class of divergence functions.
Non-negative Matrix Factorization based on γ-divergence
TLDR
Novel NMF algorithms based on the γ-divergence which is known to be robust are proposed, and robustness of proposed methods with numerical experiments is investigated.
Non-monotone projection gradient method for non-negative matrix factorization
TLDR
Experimental results show that the proposed non-monotone projection gradient method for NMF converges to better solutions than popular multiplicative update-based algorithms.
Novel Algorithm for Non-Negative Matrix Factorization
TLDR
This paper proposes an additive update algorithm that has a faster computational speed than Lee and Seung's multiplicative update algorithm for non-negative matrix factorization.
HALS-based algorithm for affine non-negative matrix factorization
  • Yifan Hou, S. Xing, Qing Xu
  • Computer Science
    2014 IEEE International Conference on Systems, Man, and Cybernetics (SMC)
  • 2014
TLDR
The well-known hierarchical alternating least squares (HALS) algorithm is adopted to solve ANMF since the update of offset vector is in the same frame of updates of factor matrices, which is quite suitable for solving ANMF.
On the Convergence of Multiplicative Update Algorithms for Nonnegative Matrix Factorization
  • Chih-Jen Lin
  • Computer Science, Mathematics
    IEEE Transactions on Neural Networks
  • 2007
TLDR
This paper proposes slight modifications of existing updates and proves their convergence, and techniques invented in this paper may be applied to prove the convergence for other bound-constrained optimization problems.
Non-negative Matrix Factorization with Sparseness Constraints
  • P. Hoyer
  • Computer Science
    J. Mach. Learn. Res.
  • 2004
TLDR
This paper shows how explicitly incorporating the notion of 'sparseness' improves the found decompositions, and provides complete MATLAB code both for standard NMF and for an extension of this technique.
Divergence Functions of Non negative Matrix Factorization: A Comparison Study
TLDR
Preliminary results show that the most familiar two divergence functions—the least squares error and the K-L divergence—are competent for unsupervised learning such as gene expression data clustering and image processing.
Symmetric Nonnegative Matrix Factorization With Beta-Divergences
TLDR
It is proved that the beta divergence sequence is monotonically convergent under this multiplicative update algorithm, capable of iteratively finding a factorization for SNMF problem by minimizing beta divergences between an input nonnegative semidefinite matrix and its SNMF approximation.
Detailed derivation of multiplicative update rules for NMF
TLDR
The goal of Non-negative Matrix Factorization is to decompose a matrix of non-negative elements into a product of two factor matrices, both of them containing also non- negative elements.
...
...

References

SHOWING 1-10 OF 26 REFERENCES
Learning the parts of objects by non-negative matrix factorization
TLDR
An algorithm for non-negative matrix factorization is demonstrated that is able to learn parts of faces and semantic features of text and is in contrast to other methods that learn holistic, not parts-based, representations.
Exponentiated Gradient Versus Gradient Descent for Linear Predictors
TLDR
The bounds suggest that the losses of the algorithms are in general incomparable, but EG(+/-) has a much smaller loss if only a few components of the input are relevant for the predictions, which is quite tight already on simple artificial data.
Additive versus exponentiated gradient updates for linear prediction
TLDR
The main methodological idea is using a distance function between weight vectors both in motivating the algorithms and as a potential function in an amortized analysis that leads to worst-case loss bounds.
Unsupervised Learning by Convex and Conic Coding
TLDR
Unsupervised learning algorithms based on convex and conic encoders based on the closest convex or conic combination of basis vectors to the input are proposed and compared with vector quantization and principal component analysis.
Aggregate and mixed-order Markov models for statistical language processing
TLDR
This work considers the use of language models whose size and accuracy are intermediate between different order n-gram models and examines smoothing procedures in which these models are interposed between different orders.
Eigenfaces for Recognition
TLDR
A near-real-time computer system that can locate and track a subject's head, and then recognize the person by comparing characteristics of the face to those of known individuals, and that is easy to implement using a neural network architecture.
A unified approach to statistical tomography using coordinate descent optimization
TLDR
This work proposes a new approach to statistically optimal image reconstruction based on direct optimization of the MAP criterion, which requires approximately the same amount of computation per iteration as EM-based approaches, but the new method converges much more rapidly.
An iterative technique for the rectification of observed distributions
J q,•.=I:Pi,Vt;, i-I f "'(~)d~=1 and "'W~O of the variation of star density along a line-of-sight from the distribution of proper motions in that direction; (2) the determination of the space
What Is the Goal of Sensory Coding?
  • D. Field
  • Computer Science
    Neural Computation
  • 1994
TLDR
It is proposed that compact coding schemes are insufficient to account for the receptive field properties of cells in the mammalian visual pathway and suggested that natural scenes, to a first approximation, can be considered as a sum of self-similar local functions (the inverse of a wavelet).
...
...