Algorithms for nonnegative matrix and tensor factorizations: a unified view based on block coordinate descent framework

@article{Kim2014AlgorithmsFN,
  title={Algorithms for nonnegative matrix and tensor factorizations: a unified view based on block coordinate descent framework},
  author={Jingu Kim and Yunlong He and Haesun Park},
  journal={Journal of Global Optimization},
  year={2014},
  volume={58},
  pages={285-319}
}
We review algorithms developed for nonnegative matrix factorization (NMF) and nonnegative tensor factorization (NTF) from a unified view based on the block coordinate descent (BCD) framework. NMF and NTF are low-rank approximation methods for matrices and tensors in which the low-rank factors are constrained to have only nonnegative elements. The nonnegativity constraints have been shown to enable natural interpretations and allow better solutions in numerous applications including text… 
Sparse nonnegative tensor decomposition using proximal algorithm and inexact block coordinate descent scheme
TLDR
The experimental results demonstrate that the proposed algorithms can efficiently impose sparsity on factor matrices, extract meaningful sparse components, and outperform state-of-the-art methods.
Sparse Nonnegative CANDECOMP/PARAFAC Decomposition in Block Coordinate Descent Framework: A Comparison Study
TLDR
This paper constructs NCP with sparse regularization (sparse NCP) by l1-norm and proposes an accelerated method to compute the objective function and relative error of sparse NCP, which has significantly improved the computation of tensor decomposition especially for higher-order tensor.
A Fast Hierarchical Alternating Least Squares Algorithm for Orthogonal Nonnegative Matrix Factorization
TLDR
This paper proposes to update the current matrix columnwisely using Hierarchical Alternating Least Squares (HALS) algorithm that is typically used for NMF, and shows that the proposed algorithm converges faster than the other conventional ONMF algorithms due to a smaller number of iterations, although the theoretical complexity is the same.
Nonnegative Matrix Factorization for Interactive Topic Modeling and Document Clustering
TLDR
In the context of clustering, this framework provides a flexible way to extend NMF such as the sparse NMF and the weakly-supervised NMF, which effectively works as the basis for the visual analytic topic modeling system that is presented.
Analysis on a Non-negative Matrix Factorization and its Applications
TLDR
This work proposes and forms a semi-smooth Newton method based on primal-dual active sets for the non-negative factorization of matrix tri-factorization and investigates its effectiveness in terms of the number of tensor products to be used in the approximation.
Novel Algorithms Based on Majorization Minimization for Nonnegative Matrix Factorization
TLDR
Two novel iterative algorithms based on Majorization Minimization are proposed - in which they formulate a novel upper bound and minimize it to get a closed form solution at every iteration and it is proved that the proposed algorithms converge to the stationary point of the problem.
Analysis on a Nonnegative Matrix Factorization and Its Applications
TLDR
A sparse low-rank approximation of positive data and images in terms of tensor products of positive vectors is proposed and a semismooth Newton method based on primal-dual active sets for the nonnegative factorization is proposed.
Semi-Orthogonal Non-Negative Matrix Factorization
TLDR
This paper proposes a semi-orthogonal NMF method that enforces one of the matrices to be orthogonal with mixed signs, thereby guarantees the rank of the factorization and preserves strict orthogonality.
SCED: A General Framework for Sparse Tensor Decomposition with Constraints and Elementwise Dynamic Learning
TLDR
This work proposes a general framework for finding the CPD of sparse tensors by modeling the sparse tensor decomposition problem by a generalized weighted CPD formulation and solving it efficiently, and is also flexible to handle constraints and dynamic data streams.
A column-wise update algorithm for nonnegative matrix factorization in Bregman divergence with an orthogonal constraint
TLDR
A column-wise update algorithm is proposed for speeding up orthogonal nonnegative matrix factorization and it is shown that the proposed algorithms converge faster than the other conventional ONMF algorithms, more than four times in the best cases, due to their smaller numbers of iterations.
...
1
2
3
4
5
...

References

SHOWING 1-10 OF 112 REFERENCES
Nonnegative matrix and tensor factorizations, least squares problems, and applications
TLDR
An accelerated block principal pivoting method is proposed to solve the NLS problems, thereby significantly speeding up the NMF and NTF computation and proposing mixed-norm regularization to promote group-level sparsity.
Fast Local Algorithms for Large Scale Nonnegative Matrix and Tensor Factorizations
  • A. Cichocki, A. Phan
  • Computer Science
    IEICE Trans. Fundam. Electron. Commun. Comput. Sci.
  • 2009
TLDR
A class of optimized local algorithms which are referred to as Hierarchical Alternating Least Squares (HALS) algorithms, which work well for NMF-based blind source separation (BSS) not only for the over-determined case but also for an under-d determined (over-complete) case if data are sufficiently sparse.
Nonnegative Matrix and Tensor Factorizations - Applications to Exploratory Multi-way Data Analysis and Blind Source Separation
This book provides a broad survey of models and efficient algorithms for Nonnegative Matrix Factorization (NMF). This includes NMFs various extensions and modifications, especially Nonnegative Tensor
Toward Faster Nonnegative Matrix Factorization: A New Algorithm and Comparisons
TLDR
This paper presents a novel algorithm for NMF based on the ANLS framework that builds upon the block principal pivoting method for the nonnegativity constrained least squares problem that overcomes some limitations of active set methods.
Fast Nonnegative Matrix Factorization: An Active-Set-Like Method and Comparisons
TLDR
A novel algorithm for NMF based on the ANLS framework that builds upon the block principal pivoting method for the nonnegativity-constrained least squares problem that overcomes a limitation of the active set method is presented.
Fast coordinate descent methods with variable selection for non-negative matrix factorization
TLDR
This paper shows that FastHals has an inefficiency in that it uses a cyclic coordinate descent scheme and thus, performs unneeded descent steps on unimportant variables, and presents a variable selection scheme that uses the gradient of the objective function to arrive at a new coordinate descent method.
Fast Nonnegative Tensor Factorization with an Active-Set-Like Method
TLDR
The block principal pivoting method overcomes some difficulties of the classical active method for the NNLS problems with a large number of variables, and techniques to accelerate the block principal pivototing method for multiple right-hand sides, which is typical in NNCP computation are introduced.
Sparse Nonnegative Matrix Factorization for Clustering
TLDR
The experimental results shows that sparse NMF does not simply provide an alternative to K-means, but rather gives much better and consistent solutions to the clustering problem, and the consistency of solutions further explains how NMF can be used to determine the unknown number of clusters from data.
Computing nonnegative tensor factorizations
TLDR
An approach for computing the NTF of a dataset that relies only on iterative linear-algebra techniques and that is comparable in cost to the nonnegative matrix factorization is described.
Nonnegative matrix factorization : complexity, algorithms and applications
TLDR
This thesis explores a closely related problem, namely nonnegative matrix factorization (NMF), a low-rank matrix approximation problem with nonnegativity constraints, and makes connections with well-known problems in graph theory, combinatorial optimization and computational geometry.
...
1
2
3
4
5
...