Online Nonnegative Matrix Factorization With Robust Stochastic Approximation

@article{Guan2012OnlineNM,
  title={Online Nonnegative Matrix Factorization With Robust Stochastic Approximation},
  author={Naiyang Guan and Dacheng Tao and Zhigang Luo and Bo Yuan},
  journal={IEEE Transactions on Neural Networks and Learning Systems},
  year={2012},
  volume={23},
  pages={1087-1099}
}
  • Naiyang Guan, D. Tao, +1 author B. Yuan
  • Published 22 May 2012
  • Mathematics, Medicine, Computer Science
  • IEEE Transactions on Neural Networks and Learning Systems
Nonnegative matrix factorization (NMF) has become a popular dimension-reduction method and has been widely applied to image processing and pattern recognition problems. However, conventional NMF learning methods require the entire dataset to reside in the memory and thus cannot be applied to large-scale or streaming datasets. In this paper, we propose an efficient online RSA-NMF algorithm (OR-NMF) that learns NMF in an incremental fashion and thus solves this problem. In particular, OR-NMF… 
NONNEGATIVE MATRIX FACTORIZATION WITH OUTLIERS
We propose an optimization framework for performing online Nonnegative Matrix Factorization (NMF) in the presence of outliers, based on `1 regularization and stochastic approximation. Due to the
Online nonnegative matrix factorization with outliers
  • Renbo Zhao, V. Tan
  • Mathematics, Computer Science
    2016 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)
  • 2016
TLDR
Numerically on a range of face datasets, the proposed optimization framework for performing online Non-negative Matrix Factorization in the presence of outliers is superior to the state-of-the-art NMF algorithms in terms of running time, basis representations and reconstruction of original images.
Online graph regularized non-negative matrix factorization for streamming data
  • Fudong Liu, Naiyang Guan, Yuhua Tang
  • Mathematics, Computer Science
    Proceedings 2014 IEEE International Conference on Security, Pattern Analysis, and Cybernetics (SPAC)
  • 2014
TLDR
This paper proposes an online GNMF (OGNMF) algorithm to process the incoming data in an incremental manner, i.e., OGNMF processes one data point or one chunk of data points one by one, which achieves better performance than the existing online NMF algorithms in terms of both accuracy and normalized mutual information.
Robust nonnegative matrix factorization with structure regularization
TLDR
A novel unsupervised learning framework, called robust structured nonnegative matrix factorization (RSNMF) which respects both global and local structures of the data space and imposes joint L2,1-norm minimization on both the loss function of NMF and the regularization of the basis matrix.
A Fast Non-Smooth Nonnegative Matrix Factorization for Learning Sparse Representation
TLDR
A fast NsNMF (FNsNMf) algorithm is proposed to speed up Ns NMF by solving a proximal function, designed based on the Lipschitz constant and can be solved through utilizing a constructed fast convergent sequence.
Online nonnegative matrix factorization based on kernel machines
  • Fei Zhu, P. Honeine
  • Mathematics, Computer Science
    2015 23rd European Signal Processing Conference (EUSIPCO)
  • 2015
TLDR
An online version of the nonlinear kernel-based NMF, where the decomposition is performed in the feature space, taking the advantage of the stochastic gradient descent and the mini-batch scheme, which has a fixed, tractable complexity independent of the increasing samples number.
Online graph regularized non-negative matrix factorization for large-scale datasets
TLDR
This paper proposes an online GNMF (OGNMF) method to process the incoming data in an incremental manner, i.e., OGNMF processes one data point or one chunk of data points one by one, and achieves better performance than the existing online NMF algorithms in terms of both accuracy and normalized mutual information.
NONNEGATIVE MATRIX FACTORIZATION BASED ON KERNEL MACHINES
Nonnegative matrix factorization (NMF) has been increasingly investigated for data analysis and dimension-reduction. To tackle large-scale data, several online techniques for NMF have been introduced
Online kernel nonnegative matrix factorization
TLDR
A framework for online nonlinear NMF, where the factorization is conducted in a kernel-induced feature space, and several general update rules are derived, in additive and multiplicative strategies, and detail the case of the Gaussian kernel.
Online Nonnegative Matrix Factorization with General Divergences
TLDR
It is proved that the sequence of learned dictionaries converges almost surely to the set of critical points of the expected loss function, by leveraging the theory of stochastic approximations and projected dynamical systems.
...
1
2
3
4
5
...

References

SHOWING 1-10 OF 48 REFERENCES
NeNMF: An Optimal Gradient Method for Nonnegative Matrix Factorization
TLDR
A new efficient NeNMF solver is presented that applies Nesterov's optimal gradient method to alternatively optimize one factor with another fixed and can be used to solve -norm, -norm and manifold regularized NMF with the optimal convergence rate.
Manifold Regularized Discriminative Nonnegative Matrix Factorization With Fast Gradient Descent
TLDR
The manifold regularization and the margin maximization to NMF are introduced and the manifold regularized discriminative NMF (MD-NMF) is obtained to overcome the aforementioned problems.
Toward Faster Nonnegative Matrix Factorization: A New Algorithm and Comparisons
  • Jingu Kim, Haesun Park
  • Mathematics, Computer Science
    2008 Eighth IEEE International Conference on Data Mining
  • 2008
TLDR
This paper presents a novel algorithm for NMF based on the ANLS framework that builds upon the block principal pivoting method for the nonnegativity constrained least squares problem that overcomes some limitations of active set methods.
Linear and Nonlinear Projective Nonnegative Matrix Factorization
TLDR
Empirical study on three real-world databases shows that PNMF can achieve the best or close to the best in clustering and run more efficiently than the compared NMF methods, especially for high-dimensional data.
Statistical Optimization of Non-Negative Matrix Factorization
TLDR
This method starts with a small subset of the columns and rows of the original matrix and uses frequentist hypothesis tests to adaptively increase the size of the problem, which achieves two objectives: 1) during the initial phase of the algorithm the authors solve far fewer, much smaller sized least squares problems and 2) all hypothesis tests failing while using all the data represents a principled, automatic stopping criterion.
Online Learning for Matrix Factorization and Sparse Coding
TLDR
A new online optimization algorithm is proposed, based on stochastic approximations, which scales up gracefully to large data sets with millions of training samples, and extends naturally to various matrix factorization formulations, making it suitable for a wide range of learning problems.
Graph Regularized Nonnegative Matrix Factorization for Data Representation
TLDR
In GNMF, an affinity graph is constructed to encode the geometrical information and a matrix factorization is sought, which respects the graph structure, and the empirical study shows encouraging results of the proposed algorithm in comparison to the state-of-the-art algorithms on real-world problems.
Efficient Document Clustering via Online Nonnegative Matrix Factorizations
TLDR
This study proposes an online NMF (ONMF) algorithm to efficiently handle very large-scale and/or streaming datasets to solve the challenge of nonnegative Matrix Factorization in the data mining and information retrieval fields.
Incremental subspace learning via non-negative matrix factorization
TLDR
Test results reported for two video applications, namely background modeling in video surveillance and clustering, demonstrate that INMF is capable of online representing data content while reducing dimension significantly.
On the Convergence of Multiplicative Update Algorithms for Nonnegative Matrix Factorization
  • Chih-Jen Lin
  • Mathematics, Computer Science
    IEEE Transactions on Neural Networks
  • 2007
TLDR
This paper proposes slight modifications of existing updates and proves their convergence, and techniques invented in this paper may be applied to prove the convergence for other bound-constrained optimization problems.
...
1
2
3
4
5
...