# Online Nonnegative Matrix Factorization With Robust Stochastic Approximation

@article{Guan2012OnlineNM, title={Online Nonnegative Matrix Factorization With Robust Stochastic Approximation}, author={Naiyang Guan and Dacheng Tao and Zhigang Luo and Bo Yuan}, journal={IEEE Transactions on Neural Networks and Learning Systems}, year={2012}, volume={23}, pages={1087-1099} }

Nonnegative matrix factorization (NMF) has become a popular dimension-reduction method and has been widely applied to image processing and pattern recognition problems. However, conventional NMF learning methods require the entire dataset to reside in the memory and thus cannot be applied to large-scale or streaming datasets. In this paper, we propose an efficient online RSA-NMF algorithm (OR-NMF) that learns NMF in an incremental fashion and thus solves this problem. In particular, OR-NMF…

## Figures, Tables, and Topics from this paper

## 273 Citations

NONNEGATIVE MATRIX FACTORIZATION WITH OUTLIERS

- 2016

We propose an optimization framework for performing online Nonnegative Matrix Factorization (NMF) in the presence of outliers, based on `1 regularization and stochastic approximation. Due to the…

Online nonnegative matrix factorization with outliers

- Mathematics, Computer Science2016 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)
- 2016

Numerically on a range of face datasets, the proposed optimization framework for performing online Non-negative Matrix Factorization in the presence of outliers is superior to the state-of-the-art NMF algorithms in terms of running time, basis representations and reconstruction of original images.

Online graph regularized non-negative matrix factorization for streamming data

- Mathematics, Computer ScienceProceedings 2014 IEEE International Conference on Security, Pattern Analysis, and Cybernetics (SPAC)
- 2014

This paper proposes an online GNMF (OGNMF) algorithm to process the incoming data in an incremental manner, i.e., OGNMF processes one data point or one chunk of data points one by one, which achieves better performance than the existing online NMF algorithms in terms of both accuracy and normalized mutual information.

Robust nonnegative matrix factorization with structure regularization

- Computer ScienceNeurocomputing
- 2020

A novel unsupervised learning framework, called robust structured nonnegative matrix factorization (RSNMF) which respects both global and local structures of the data space and imposes joint L2,1-norm minimization on both the loss function of NMF and the regularization of the basis matrix.

A Fast Non-Smooth Nonnegative Matrix Factorization for Learning Sparse Representation

- Mathematics, Computer ScienceIEEE Access
- 2016

A fast NsNMF (FNsNMf) algorithm is proposed to speed up Ns NMF by solving a proximal function, designed based on the Lipschitz constant and can be solved through utilizing a constructed fast convergent sequence.

Online nonnegative matrix factorization based on kernel machines

- Mathematics, Computer Science2015 23rd European Signal Processing Conference (EUSIPCO)
- 2015

An online version of the nonlinear kernel-based NMF, where the decomposition is performed in the feature space, taking the advantage of the stochastic gradient descent and the mini-batch scheme, which has a fixed, tractable complexity independent of the increasing samples number.

Online graph regularized non-negative matrix factorization for large-scale datasets

- Computer ScienceNeurocomputing
- 2016

This paper proposes an online GNMF (OGNMF) method to process the incoming data in an incremental manner, i.e., OGNMF processes one data point or one chunk of data points one by one, and achieves better performance than the existing online NMF algorithms in terms of both accuracy and normalized mutual information.

NONNEGATIVE MATRIX FACTORIZATION BASED ON KERNEL MACHINES

- 2015

Nonnegative matrix factorization (NMF) has been increasingly investigated for data analysis and dimension-reduction. To tackle large-scale data, several online techniques for NMF have been introduced…

Online kernel nonnegative matrix factorization

- Computer Science, MathematicsSignal Process.
- 2017

A framework for online nonlinear NMF, where the factorization is conducted in a kernel-induced feature space, and several general update rules are derived, in additive and multiplicative strategies, and detail the case of the Gaussian kernel.

Online Nonnegative Matrix Factorization with General Divergences

- Mathematics, Computer ScienceAISTATS
- 2017

It is proved that the sequence of learned dictionaries converges almost surely to the set of critical points of the expected loss function, by leveraging the theory of stochastic approximations and projected dynamical systems.

## References

SHOWING 1-10 OF 48 REFERENCES

NeNMF: An Optimal Gradient Method for Nonnegative Matrix Factorization

- Mathematics, Computer ScienceIEEE Transactions on Signal Processing
- 2012

A new efficient NeNMF solver is presented that applies Nesterov's optimal gradient method to alternatively optimize one factor with another fixed and can be used to solve -norm, -norm and manifold regularized NMF with the optimal convergence rate.

Manifold Regularized Discriminative Nonnegative Matrix Factorization With Fast Gradient Descent

- Mathematics, MedicineIEEE Transactions on Image Processing
- 2011

The manifold regularization and the margin maximization to NMF are introduced and the manifold regularized discriminative NMF (MD-NMF) is obtained to overcome the aforementioned problems.

Toward Faster Nonnegative Matrix Factorization: A New Algorithm and Comparisons

- Mathematics, Computer Science2008 Eighth IEEE International Conference on Data Mining
- 2008

This paper presents a novel algorithm for NMF based on the ANLS framework that builds upon the block principal pivoting method for the nonnegativity constrained least squares problem that overcomes some limitations of active set methods.

Linear and Nonlinear Projective Nonnegative Matrix Factorization

- Mathematics, MedicineIEEE Transactions on Neural Networks
- 2010

Empirical study on three real-world databases shows that PNMF can achieve the best or close to the best in clustering and run more efficiently than the compared NMF methods, especially for high-dimensional data.

Statistical Optimization of Non-Negative Matrix Factorization

- Mathematics, Computer ScienceAISTATS
- 2011

This method starts with a small subset of the columns and rows of the original matrix and uses frequentist hypothesis tests to adaptively increase the size of the problem, which achieves two objectives: 1) during the initial phase of the algorithm the authors solve far fewer, much smaller sized least squares problems and 2) all hypothesis tests failing while using all the data represents a principled, automatic stopping criterion.

Online Learning for Matrix Factorization and Sparse Coding

- Computer Science, MathematicsJ. Mach. Learn. Res.
- 2010

A new online optimization algorithm is proposed, based on stochastic approximations, which scales up gracefully to large data sets with millions of training samples, and extends naturally to various matrix factorization formulations, making it suitable for a wide range of learning problems.

Graph Regularized Nonnegative Matrix Factorization for Data Representation

- Computer ScienceIEEE Transactions on Pattern Analysis and Machine Intelligence
- 2011

In GNMF, an affinity graph is constructed to encode the geometrical information and a matrix factorization is sought, which respects the graph structure, and the empirical study shows encouraging results of the proposed algorithm in comparison to the state-of-the-art algorithms on real-world problems.

Efficient Document Clustering via Online Nonnegative Matrix Factorizations

- Computer ScienceSDM
- 2011

This study proposes an online NMF (ONMF) algorithm to efficiently handle very large-scale and/or streaming datasets to solve the challenge of nonnegative Matrix Factorization in the data mining and information retrieval fields.

Incremental subspace learning via non-negative matrix factorization

- Computer Science, MathematicsPattern Recognit.
- 2009

Test results reported for two video applications, namely background modeling in video surveillance and clustering, demonstrate that INMF is capable of online representing data content while reducing dimension significantly.

On the Convergence of Multiplicative Update Algorithms for Nonnegative Matrix Factorization

- Mathematics, Computer ScienceIEEE Transactions on Neural Networks
- 2007

This paper proposes slight modifications of existing updates and proves their convergence, and techniques invented in this paper may be applied to prove the convergence for other bound-constrained optimization problems.