Accelerated parallel and distributed algorithm using limited internal memory for nonnegative matrix factorization

  title={Accelerated parallel and distributed algorithm using limited internal memory for nonnegative matrix factorization},
  author={Duy Khuong Nguyen and Tu Bao Ho},
  journal={Journal of Global Optimization},
  • D. Nguyen, T. Ho
  • Published 29 June 2015
  • Computer Science
  • Journal of Global Optimization
Nonnegative matrix factorization (NMF) is a powerful technique for dimension reduction, extracting latent factors and learning part-based representation. For large datasets, NMF performance depends on some major issues such as fast algorithms, fully parallel distributed feasibility and limited internal memory. This research designs a fast fully parallel and distributed algorithm using limited internal memory to reach high NMF performance for large datasets. Specially, we propose a flexible… 
BALS: Blocked Alternating Least Squares for Parallel Sparse Matrix Factorization on GPUs
This article presents an efficient implementation of the alternative least squares (ALS) algorithm called BALSbuilt on top of a new sparse matrix format for parallel matrix factorization, and proposes a data reordering technique to sort sparse matrices according to nonzeros.
Final Report for AOARD Grant FA 2386-15-1-4006 “ Dimensionality Reduction in Big Data with Nonnegative Matrix Factorization ” Name of Principal Investigator :
  • T. Ho
  • Computer Science
  • 2017
The experiments indicate that the proposed model and algorithm outperform the NMF model and its state-of-the-art algorithms and can achieve high sparse representation with higher accuracy in comparison with similar approaches.
Distributed geometric nonnegative matrix factorization and hierarchical alternating least squares–based nonnegative tensor factorization with the MapReduce paradigm
This study extends the previously reported distributed hierarchical alternating least squares algorithm to the multi‐way array factorization model, where it is assumed that the observed multi‐ way data can be partitioned into chunks along one mode.


Toward Faster Nonnegative Matrix Factorization: A New Algorithm and Comparisons
This paper presents a novel algorithm for NMF based on the ANLS framework that builds upon the block principal pivoting method for the nonnegativity constrained least squares problem that overcomes some limitations of active set methods.
Limited-Memory Fast Gradient Descent Method for Graph Regularized Nonnegative Matrix Factorization
An efficient limited-memory FGD (L-FGD) method is proposed for optimizing graph regularized nonnegative matrix factorization and its clustering performance is validated for optimizing KL-divergence based GNMF on two popular face image datasets including ORL and PIE and two text corpora.
Accelerated Multiplicative Updates and Hierarchical ALS Algorithms for Nonnegative Matrix Factorization
A simple way to significantly accelerate two well-known algorithms designed to solve NMF problems: the multiplicative updates of Lee and Seung and the hierarchical alternating least squares of Cichocki et al. is proposed.
Large-scale matrix factorization with distributed stochastic gradient descent
A novel algorithm to approximately factor large matrices with millions of rows, millions of columns, and billions of nonzero elements, called DSGD, that can be fully distributed and run on web-scale datasets using, e.g., MapReduce.
Fast coordinate descent methods with variable selection for non-negative matrix factorization
This paper shows that FastHals has an inefficiency in that it uses a cyclic coordinate descent scheme and thus, performs unneeded descent steps on unimportant variables, and presents a variable selection scheme that uses the gradient of the objective function to arrive at a new coordinate descent method.
Nonnegative Matrix Factorization Based on Alternating Nonnegativity Constrained Least Squares and Active Set Method
This paper introduces an algorithm for NMF based on alternating nonnegativity constrained least squares (NMF/ANLS) and the active set-based fast algorithm for nonNegativity constrained most squares with multiple right-hand side vectors, and discusses its convergence properties and a rigorous convergence criterion based on the Karush-Kuhn-Tucker (KKT) conditions.
NeNMF: An Optimal Gradient Method for Nonnegative Matrix Factorization
A new efficient NeNMF solver is presented that applies Nesterov's optimal gradient method to alternatively optimize one factor with another fixed and can be used to solve -norm, -norm and manifold regularized NMF with the optimal convergence rate.
Algorithms for nonnegative matrix and tensor factorizations: a unified view based on block coordinate descent framework
Algorithms developed for nonnegative matrix factorization and nonnegative tensor factorization are reviewed from a unified view based on the block coordinate descent (BCD) framework to propose efficient algorithms for updating NMF when there is a small change in the reduced dimension or in the data.
Nonnegative Matrix Factorization: Models, Algorithms and Applications
This chapter surveys NMF in terms of the model formulation and its variations and extensions, algorithms and applications, as well as its relations with K-means and Probabilistic Latent Semantic Indexing (PLSI).
Non-negative Matrix Factorization with Quasi-Newton Optimization
This work derived a relatively simple second-order quasi-Newton method for NMF: so-called Amari alpha divergence, which has been extensively tested for blind source separation problems, both for signals and images.