# On the Convergence of Multiplicative Update Algorithms for Nonnegative Matrix Factorization

@article{Lin2007OnTC, title={On the Convergence of Multiplicative Update Algorithms for Nonnegative Matrix Factorization}, author={Chih-Jen Lin}, journal={IEEE Transactions on Neural Networks}, year={2007}, volume={18}, pages={1589-1596} }

Nonnegative matrix factorization (NMF) is useful to find basis information of nonnegative data. Currently, multiplicative updates are a simple and popular way to find the factorization. However, for the common NMF approach of minimizing the Euclidean distance between approximate and true values, no proof has shown that multiplicative updates converge to a stationary point of the NMF optimization problem. Stationarity is important as it is a necessary condition of a local minimum. This paper…

## Tables from this paper

## 402 Citations

A Fast Algorithm for Nonnegative Matrix Factorization and Its Convergence

- Computer ScienceIEEE Transactions on Neural Networks and Learning Systems
- 2014

A new multiplicative update algorithm for minimizing the Euclidean distance between approximate and true values is proposed based on the optimization principle and the auxiliary function method and it is proved that this new algorithm not only converges to a stationary point, but also does faster than existing ones.

A Modified Multiplicative Update Algorithm for Euclidean Distance-Based Nonnegative Matrix Factorization and Its Global Convergence

- Computer ScienceICONIP
- 2011

This paper considers NMF in which the approximation error is measured by the Euclidean distance between two matrices and proposes a modified multiplicative update algorithm in order to overcome the drawback of the original version and prove its global convergence.

A unified convergence analysis of the multiplicative update algorithm for nonnegative matrix factorization

- Computer Science2017 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)
- 2017

This work provides a conceptually simple, self-contained, and unified proof for the convergence of the MU algorithm applied on NMF with a wide range of divergences and regularizations.

Global convergence of modified multiplicative updates for nonnegative matrix factorization

- Mathematics, Computer ScienceComput. Optim. Appl.
- 2014

Using Zangwill’s global convergence theorem, it is proved that any sequence of solutions generated by either of those modified updates has at least one convergence subsequence and the limit of any convergent subsequence is a stationary point of the corresponding optimization problem.

A Unified Convergence Analysis of the Multiplicative Update Algorithm for Regularized Nonnegative Matrix Factorization

- Computer ScienceIEEE Transactions on Signal Processing
- 2018

This work provides a conceptually simple, self-contained, and unified proof for the convergence of the MU algorithm applied on NMF with a wide range of divergences and regularizers.

Global convergence of a modified HALS algorithm for nonnegative matrix factorization

- Computer Science, Mathematics2015 IEEE 6th International Workshop on Computational Advances in Multi-Sensor Adaptive Processing (CAMSAP)
- 2015

This paper considers the HALS algorithm for the Frobenius norm-based NMF, and proves that a modified version has the global convergence property in the sense of Zangwill.

Stability analysis of multiplicative update algorithms for non-negative matrix factorization

- Computer Science2011 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)
- 2011

It is shown that Lyapunov's stability theory provides a very enlightening viewpoint on the problem of NMF multiplicative update and is proved the stability of supervised NMF and study the more difficult case of unsupervised NMF.

Stability Analysis of Multiplicative Update Algorithms and Application to Nonnegative Matrix Factorization

- Computer ScienceIEEE Transactions on Neural Networks
- 2010

It is shown that Lyapunov's stability theory provides a very enlightening viewpoint on the problem of NMF multiplicative update, and the exponential or asymptotic stability of the solutions to general optimization problems with nonnegative constraints is proved.

Multiplicative update for a class of constrained optimization problems related to NMF and its global convergence

- Mathematics2016 24th European Signal Processing Conference (EUSIPCO)
- 2016

It is proved that a modified version of the update rule has the global convergence property in the sense of Zangwill under certain conditions and can be applied to many existing multiplicative update rules for NMF to guarantee their global convergence.

A unified global convergence analysis of multiplicative update rules for nonnegative matrix factorization

- Computer Science, MathematicsComput. Optim. Appl.
- 2018

This paper provides a sufficient condition for a general multiplicative update rule to have the global convergence property in the sense that any sequence of solutions has at least one convergent subsequence and the limit of any convergence subsequence is a stationary point of the optimization problem.

## References

SHOWING 1-10 OF 19 REFERENCES

Algorithms for Non-negative Matrix Factorization

- Computer ScienceNIPS
- 2000

Two different multiplicative algorithms for non-negative matrix factorization are analyzed and one algorithm can be shown to minimize the conventional least squares error while the other minimizes the generalized Kullback-Leibler divergence.

Projected Gradient Methods for Nonnegative Matrix Factorization

- Computer ScienceNeural Computation
- 2007

This letter proposes two projected gradient methods for nonnegative matrix factorization, both of which exhibit strong optimization properties and discuss efficient implementations and demonstrate that one of the proposed methods converges faster than the popular multiplicative update approach.

Nonnegative matrix factorization and I-divergence alternating minimization☆

- Computer Science
- 2006

Optimality, computation, and interpretation of nonnegative matrix factorizations

- Computer Science
- 2004

The theoretical Kuhn-Tucker optimality condition is described in explicit form and a number of numerical techniques, old and new, are suggested for the nonnegative matrix factorization problems.

Algorithms and applications for approximate nonnegative matrix factorization

- Computer ScienceComput. Stat. Data Anal.
- 2007

Non-negative Matrix Factorization with Sparseness Constraints

- Computer ScienceJ. Mach. Learn. Res.
- 2004

This paper shows how explicitly incorporating the notion of 'sparseness' improves the found decompositions, and provides complete MATLAB code both for standard NMF and for an extension of this technique.

On reduced rank nonnegative matrix factorization for symmetric nonnegative matrices

- Mathematics, Computer Science
- 2004

Accelerating the Lee-Seung Algorithm for Nonnegative Matrix Factorization

- Computer Science
- 2005

A variation of one of the Lee-Seung algorithms with a notably improved performance is presented and it is shown that algorithms of this type do not necessarily converge to local minima.

Positive matrix factorization: A non-negative factor model with optimal utilization of error estimates of data values†

- Mathematics
- 1994

A new variant ‘PMF’ of factor analysis is described. It is assumed that X is a matrix of observed data and σ is the known matrix of standard deviations of elements of X. Both X and σ are of…

Non-negative sparse coding

- Computer ScienceProceedings of the 12th IEEE Workshop on Neural Networks for Signal Processing
- 2002

A simple yet efficient multiplicative algorithm for finding the optimal values of the hidden components of non-negative sparse coding and how the basis vectors can be learned from the observed data is shown.