# A Generalized Divergence Measure for Nonnegative Matrix Factorization

@article{Kompass2007AGD, title={A Generalized Divergence Measure for Nonnegative Matrix Factorization}, author={Raul Kompass}, journal={Neural Computation}, year={2007}, volume={19}, pages={780-791} }

This letter presents a general parametric divergence measure. The metric includes as special cases quadratic error and Kullback-Leibler divergence. A parametric generalization of the two different multiplicative update rules for nonnegative matrix factorization by Lee and Seung (2001) is shown to lead to locally optimal solutions of the nonnegative matrix factorization problem with this new cost function. Numeric simulations demonstrate that the new update rule may improve the quadratic…

## 207 Citations

Convergence-guaranteed multiplicative algorithms for nonnegative matrix factorization with β-divergence

- Computer Science2010 IEEE International Workshop on Machine Learning for Signal Processing
- 2010

This paper presents a new multiplicative algorithm for nonnegative matrix factorization with β-divergence, theoretically proven for any real-valued β based on the auxiliary function method.

A unified global convergence analysis of multiplicative update rules for nonnegative matrix factorization

- Computer Science, MathematicsComput. Optim. Appl.
- 2018

This paper provides a sufficient condition for a general multiplicative update rule to have the global convergence property in the sense that any sequence of solutions has at least one convergent subsequence and the limit of any convergence subsequence is a stationary point of the optimization problem.

Algorithms for Nonnegative Matrix Factorization with the β-Divergence

- Computer ScienceNeural Computation
- 2011

This letter describes algorithms for nonnegative matrix factorization (NMF) with the β-divergence (β-NMF). The β-divergence is a family of cost functions parameterized by a single shape parameter β…

A family of modified projective Nonnegative Matrix Factorization algorithms

- Computer Science2007 9th International Symposium on Signal Processing and Its Applications
- 2007

Experimental results show that versions of P-NMF derive bases which are somewhat better suitable for a localized and sparse representation than NMF, as well as being more orthogonal.

Algorithms for nonnegative matrix factorization with the beta-divergence

- Computer ScienceArXiv
- 2010

This paper describes algorithms for nonnegative matrix factorization (NMF) with the beta-divergence (beta-NMF). The beta-divergence is a family of cost functions parametrized by a single shape…

Unified Development of Multiplicative Algorithms for Linear and Quadratic Nonnegative Matrix Factorization

- Computer ScienceIEEE Transactions on Neural Networks
- 2011

A general approach to derive the auxiliary function for a wide variety of NMF problems, as long as the approximation objective can be expressed as a finite sum of monomials with real exponents is proposed.

Stability analysis of multiplicative update algorithms for non-negative matrix factorization

- Computer Science2011 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)
- 2011

It is shown that Lyapunov's stability theory provides a very enlightening viewpoint on the problem of NMF multiplicative update and is proved the stability of supervised NMF and study the more difficult case of unsupervised NMF.

Stability Analysis of Multiplicative Update Algorithms and Application to Nonnegative Matrix Factorization

- Computer ScienceIEEE Transactions on Neural Networks
- 2010

It is shown that Lyapunov's stability theory provides a very enlightening viewpoint on the problem of NMF multiplicative update, and the exponential or asymptotic stability of the solutions to general optimization problems with nonnegative constraints is proved.

Nonnegative matrix factorizations as probabilistic inference in composite models

- Computer Science2009 17th European Signal Processing Conference
- 2009

This paper describes multiplicative, Expectation-Maximization, Markov chain Monte Carlo and Variational Bayes algorithms for the NMF problem, and aims at providing statistical insights to NMF.

## References

SHOWING 1-9 OF 9 REFERENCES

Algorithms for Non-negative Matrix Factorization

- Computer ScienceNIPS
- 2000

Two different multiplicative algorithms for non-negative matrix factorization are analyzed and one algorithm can be shown to minimize the conventional least squares error while the other minimizes the generalized Kullback-Leibler divergence.

Non-negative matrix factorization with alpha-divergence

- Computer SciencePattern Recognit. Lett.
- 2008

On alpha-divergence based nonnegative matrix factorization for clustering cancer gene expression data

- Computer ScienceArtif. Intell. Medicine
- 2008

Learning the parts of objects by non-negative matrix factorization

- Computer ScienceNature
- 1999

An algorithm for non-negative matrix factorization is demonstrated that is able to learn parts of faces and semantic features of text and is in contrast to other methods that learn holistic, not parts-based, representations.

Neural Networks, Principal Components, and Subspaces

- Computer ScienceInt. J. Neural Syst.
- 1989

A single neuron with Hebbian-type learning for the connection weights, and with nonlinear internal feedback, has been shown to extract the statistical principal components of its stationary input pattern sequence, which yields a multi-dimensional, principal component subspace.

Backpropagation Applied to Handwritten Zip Code Recognition

- Computer ScienceNeural Computation
- 1989

This paper demonstrates how constraints from the task domain can be integrated into a backpropagation network through the architecture of the network, successfully applied to the recognition of handwritten zip code digits provided by the U.S. Postal Service.

Preintegration Lateral Inhibition Enhances Unsupervised Learning

- Computer Science, BiologyNeural Computation
- 2002

It is argued that preintegration lateral inhibition has computational advantages over conventional neural network architectures while remaining equally biologically plausible.

Maximum likelihood from incomplete data via the EM - algorithm plus discussions on the paper

- Engineering
- 1977

Vibratory power unit for vibrating conveyers and screens comprising an asynchronous polyphase motor, at least one pair of associated unbalanced masses disposed on the shaft of said motor, with the…

CBCL face database #1

- Available online at http://cbcl.mit.edu/cbcl/software-datasets/facedata2.html.
- 2000