# Nonnegative Matrix and Tensor Factorization [Lecture Notes]

@article{Cichocki2008NonnegativeMA, title={Nonnegative Matrix and Tensor Factorization [Lecture Notes]}, author={Andrzej Cichocki and Rafał Zdunek and Shun‐ichi Amari}, journal={IEEE Signal Processing Magazine}, year={2008}, volume={25}, pages={142-145} }

In these lecture notes, the authors have outlined several approaches to solve a NMF/NTF problem. The following main conclusions can be drawn: 1) Multiplicative algorithms are not necessary the best approaches for NMF, especially if data representations are not very redundant or sparse. 2) Much better performance can be achieved using the FP-ALS (especially for large-scale problems), IPC, and QN methods. 3) To achieve high performance it is quite important to use the multilayer structure with…

## 142 Citations

A multilevel approach for nonnegative matrix factorization

- Computer ScienceJ. Comput. Appl. Math.
- 2012

Descent methods for Nonnegative Matrix Factorization

- Computer Science, MathematicsArXiv
- 2008

By interpreting this method as a rank-one approximation of the residue matrix, it is proved that it \(converges\) and also extend it to the nonnegative tensor factorization and introduce some variants of the method by imposing some additional controllable constraints such as: sparsity, discreteness and smoothness.

Blind source separation by nonnegative matrix factorization with minimum-volume constraint

- Computer Science2010 International Conference on Intelligent Control and Information Processing
- 2010

A minimum-volume constrained NMF is proposed and an efficient multiplicative update algorithm is developed based on the natural gradient optimization that can be applied to the blind source separation (BSS) problem.

Projected Gradient Methods for Nonnegative Matrix Factorization

- Computer ScienceNeural Computation
- 2007

This letter proposes two projected gradient methods for nonnegative matrix factorization, both of which exhibit strong optimization properties and discuss efficient implementations and demonstrate that one of the proposed methods converges faster than the popular multiplicative update approach.

A Generalized Hierarchical Nonnegative Tensor Decomposition

- Computer ScienceArXiv
- 2021

A new HNTF model is proposed which directly generalizes a HNMF model special case, and a supervised extension is provided to provide a multiplicative updates training method for this model.

Fast Nonnegative Matrix Factorization Algorithms Using Projected Gradient Approaches for Large-Scale Problems

- Computer ScienceComput. Intell. Neurosci.
- 2008

This paper investigates and test some recent PG methods in the context of their applicability to NMF, and focuses on the following modified methods: projected Landweber, Barzilai-Borwein gradient projection, projected sequential subspace optimization, interior-point Newton (IPN), and sequential coordinate-wise.

A proximal ANLS algorithm for nonnegative tensor factorization with a periodic enhanced line search

- Computer Science
- 2013

A Proximal ANLS (PANLS) algorithm to enforce convergence and to speed up the PANLS method, it is proposed to combine it with a periodic enhanced line search strategy and the resulting algorithm, PANLS/PELS, converges to a critical point of the nonnegative tensor factorization problem under mild conditions.

Music Enhancement Using Nonnegative Matrix Factorization with Penalty Masking

- Computer Science2013 IEEE 16th International Conference on Computational Science and Engineering
- 2013

The implementation of nonnegative matrix factorization is carried out on a stepwise basis and a perception based filtering as the preprocess for more reliable noise detection and one example of numerical design based on a reverberant room recording is shown to demonstrate the usefulness of this approach.

Minimax Lower Bounds for Nonnegative Matrix Factorization

- Computer Science2018 IEEE Statistical Signal Processing Workshop (SSP)
- 2018

This paper provides lower bounds on the minimax risk (the minimum achievable worst case mean squared error) of estimating the non-negative dictionary matrix under a set of locality and statistical assumptions.

Solving Time-Domain Audio Inverse Problems Using Nonnegative Tensor Factorization

- Computer ScienceIEEE Transactions on Signal Processing
- 2018

A new algorithm based on the NMF (and NTF) in the short-time Fourier domain is proposed for solving a large class of audio inverse problems with missing or corrupted time-domain samples.

## References

SHOWING 1-10 OF 11 REFERENCES

Projected Gradient Methods for Nonnegative Matrix Factorization

- Computer ScienceNeural Computation
- 2007

This letter proposes two projected gradient methods for nonnegative matrix factorization, both of which exhibit strong optimization properties and discuss efficient implementations and demonstrate that one of the proposed methods converges faster than the popular multiplicative update approach.

Novel Multi-layer Non-negative Tensor Factorization with Sparsity Constraints

- Computer ScienceICANNGA
- 2007

In this paper we present a new method of 3D non-negative tensor factorization (NTF) that is robust in the presence of noise and has many potential applications, including multi-way blind source…

Generalized Nonnegative Matrix Approximations with Bregman Divergences

- Computer ScienceNIPS
- 2005

This paper makes algorithmic progress by modeling and solving (using multiplicative updates) new generalized NNMA problems that minimize Bregman divergences between the input matrix and its low-rank approximation.

Hierarchical ALS Algorithms for Nonnegative Matrix and 3D Tensor Factorization

- Computer ScienceICA
- 2007

This paper proposes to use local cost functions whose simultaneous or sequential (one by one) minimization leads to a very simple ALS algorithm which works under some sparsity constraints both for an under-determined and overdetermined model.

Nonnegative matrix factorization with constrained second-order optimization

- Computer ScienceSignal Process.
- 2007

Algorithms and applications for approximate nonnegative matrix factorization

- Computer ScienceComput. Stat. Data Anal.
- 2007

Extended SMART Algorithms for Non-negative Matrix Factorization

- Computer ScienceICAISC
- 2006

A family of new extended SMART (Simultaneous Multiplicative Algebraic Reconstruction Technique) algorithms for Non-negative Matrix Factorization (NMF) are derived by improved efficiency and convergence rate and can be applied for various distributions of data and additive noise.

Learning the parts of objects by non-negative matrix factorization

- Computer ScienceNature
- 1999

An algorithm for non-negative matrix factorization is demonstrated that is able to learn parts of faces and semantic features of text and is in contrast to other methods that learn holistic, not parts-based, representations.

Nonnegative matrix factorization for rapid recovery of constituent spectra in magnetic resonance chemical shift imaging of the brain

- Computer ScienceIEEE Transactions on Medical Imaging
- 2004

An algorithm for blindly recovering constituent source spectra from magnetic resonance (MR) chemical shift imaging (CSI) of the human brain is presented, showing that it can be used to recover tissue-specific spectra given a processing hierarchy that proceeds coarse-to-fine.

Metagenes and molecular pattern discovery using matrix factorization

- Computer Science, BiologyProceedings of the National Academy of Sciences of the United States of America
- 2004

Nonnegative matrix factorization is described, an algorithm based on decomposition by parts that can reduce the dimension of expression data from thousands of genes to a handful of metagenes, and found less sensitive to a priori selection of genes or initial conditions and able to detect alternative or context-dependent patterns of gene expression in complex biological systems.