A high-performance parallel algorithm for nonnegative matrix factorization

  title={A high-performance parallel algorithm for nonnegative matrix factorization},
  author={Ramakrishnan Kannan and Grey Ballard and Haesun Park},
  journal={Proceedings of the 21st ACM SIGPLAN Symposium on Principles and Practice of Parallel Programming},
  • R. KannanGrey BallardHaesun Park
  • Published 30 September 2015
  • Computer Science
  • Proceedings of the 21st ACM SIGPLAN Symposium on Principles and Practice of Parallel Programming
Non-negative matrix factorization (NMF) is the problem of determining two non-negative low rank factors W and H, for the given input matrix A, such that A ≈ WH. NMF is a useful tool for many applications in different domains such as topic modeling in text mining, background separation in video analysis, and community detection in social networks. Despite its popularity in the data mining community, there is a lack of efficient distributed algorithms to solve the problem for big data sets. We… 

Figures and Tables from this paper

Partitioning and Communication Strategies for Sparse Non-negative Matrix Factorization

This paper focuses on scaling algorithms for NMF to very large sparse datasets and massively parallel machines by employing effective algorithms, communication patterns, and partitioning schemes that leverage the sparsity of the input matrix.

An Efficient Algorithm for Non-Negative Matrix Factorization with Random Projections

This work applies a random compression scheme to drastically reduce the dimensionality of the problem, preserving well the pairwise distances between data points and inherently limiting the memory and communication load.

DSANLS: Accelerating Distributed Nonnegative Matrix Factorization via Sketching

This paper proposes a distributed sketched alternating nonnegative least squares (DSANLS) framework for NMF, which utilizes a matrix sketching technique to reduce the size of non negative least squares subproblems in each iteration for U and V.

PL-NMF: Parallel Locality-Optimized Non-negative Matrix Factorization

A parallel NMF algorithm based on the HALS (Hierarchical Alternating Least Squares) scheme that incorporates algorithmic transformations to enhance data locality is devised, demonstrating significant performance improvement over existing state-of-the-art parallelNMF algorithms.

ALO-NMF: Accelerated Locality-Optimized Non-negative Matrix Factorization

A novel optimization method for parallel NMF algorithm based on the HALS (Hierarchical Alternating Least Squares) scheme that incorporates algorithmic transformations to enhance data locality is presented, demonstrating a new Accelerated Locality-Optimized NMF (ALO-NMF).

Parallel Hierarchical Clustering using Rank-Two Nonnegative Matrix Factorization

A parallel algorithm for hierarchical clustering that uses a divide-and-conquer approach based on rank-two NMF to split a data set into two cohesive parts, finding more structure in the data than a flat NMF clustering.

GPU-accelerated Large-Scale Non-negative Matrix Factorization Using Spark

A parallel algorithm based on GPU for NMF in Spark platform is proposed, which makes full use of the advantages of in-memory computation mode and GPU Single-Instruction Multiple-data Streams mode and can effectively deal with the non-negative decomposition of higher-order matrices, which greatly improves the computational efficiency.

Parallelization of the Hierarchical Alternating Least Squares Algorithm for Nonnegative Matrix Factorization

  • M. FlatzR. KutilM. Vajtersic
  • Computer Science
    2018 IEEE 4th International Forum on Research and Technology for Society and Industry (RTSI)
  • 2018
It is shown that a parallelization strategy similar to ANLS parallelizations exists and yields good speedups for up to 64 processes and satisfactory beyond and are competitive in comparison to previous solutions to the NMF problem.

High Performance Parallel Algorithms for Tensor Decompositions

The main focus of this thesis is on efficient decomposition of high dimensional sparse tensors, with hundreds of millions to billions of nonzero entries, which arise in many emerging big data applications and introduces a tree-based computational scheme that carries out expensive operations faster by factoring out and storing common partial results and effectivelyre-using them.

PLANC: Parallel Low Rank Approximation with Non-negativity Constraints

This work proposes a distributed-memory parallel computing solution to handle massive data sets, loading the input data across the memories of multiple nodes and performing efficient and scalable parallel algorithms to compute the low-rank approximation.



Fast Nonnegative Matrix Factorization: An Active-Set-Like Method and Comparisons

A novel algorithm for NMF based on the ANLS framework that builds upon the block principal pivoting method for the nonnegativity-constrained least squares problem that overcomes a limitation of the active set method is presented.

Scalable Nonnegative Matrix Factorization with Block-wise Updates

By leveraging a new form of update functions, this paper can perform local aggregation and fully explore parallelism, and perform frequent updates, which aim to use the most recently updated data whenever possible, and are more efficient than their traditional concurrent counterparts.

NMF-mGPU: non-negative matrix factorization on multi-GPU systems

NMF-mGPU is an efficient and easy-to-use implementation of the NMF algorithm that takes advantage of the high computing performance delivered by Graphics-Processing Units(GPUs) and can be used "out of the box" by researchers with little or no expertise in GPU programming in a variety of platforms.

Symmetric Nonnegative Matrix Factorization for Graph Clustering

Symmetric NMF is proposed as a general framework for graph clustering, which inherits the advantages of NMF by enforcing nonnegativity on the clustering assignment matrix, and serves as a potential basis for many extensions.

Sparse non-negative matrix factorizations via alternating non-negativity-constrained least squares for microarray data analysis

The experimental results illustrate that the proposed sparse NMF algorithm often achieves better clustering performance with shorter computing time compared to other existing NMF algorithms.

Scalable sparse tensor decompositions in distributed memory systems

  • O. KayaB. Uçar
  • Computer Science
    SC15: International Conference for High Performance Computing, Networking, Storage and Analysis
  • 2015
A distributed memory sparse tensor library, HyperTensor, is designed, which implements a well-known algorithm for the CANDECOMP-/PARAFAC (CP) tensor decomposition using the task definitions and the associated partitioning methods.

Nonnegative Matrix and Tensor Factorizations - Applications to Exploratory Multi-way Data Analysis and Blind Source Separation

This book provides a broad survey of models and efficient algorithms for Nonnegative Matrix Factorization (NMF). This includes NMFs various extensions and modifications, especially Nonnegative Tensor

Non-negative Matrix Factorization with Sparseness Constraints

  • P. Hoyer
  • Computer Science
    J. Mach. Learn. Res.
  • 2004
This paper shows how explicitly incorporating the notion of 'sparseness' improves the found decompositions, and provides complete MATLAB code both for standard NMF and for an extension of this technique.

Large-scale matrix factorization with distributed stochastic gradient descent

A novel algorithm to approximately factor large matrices with millions of rows, millions of columns, and billions of nonzero elements, called DSGD, that can be fully distributed and run on web-scale datasets using, e.g., MapReduce.

Algorithms for Non-negative Matrix Factorization

Two different multiplicative algorithms for non-negative matrix factorization are analyzed and one algorithm can be shown to minimize the conventional least squares error while the other minimizes the generalized Kullback-Leibler divergence.