Online Learning for Matrix Factorization and Sparse Coding

Abstract

Sparse coding—that is, modelling data vectors as sparse linear combinations of basis elements—is widely used in machine learning, neuroscience, signal processing, and statistics. This paper fo-cuses on the large-scale matrix factorization problem that consists of learning the basis set in order to adapt it to specific data. Variations of this problem include dictionary learning in signal processing , non-negative matrix factorization and sparse principal component analysis. In this paper, we propose to address these tasks with a new online optimization algorithm, based on stochastic approximations, which scales up gracefully to large data sets with millions of training samples, and extends naturally to various matrix factorization formulations, making it suitable for a wide range of learning problems. A proof of convergence is presented, along with experiments with natural images and genomic data demonstrating that it leads to state-of-the-art performance in terms of speed and optimization for both small and large data sets.

DOI: 10.1145/1756006.1756008

Extracted Key Phrases

7 Figures and Tables

Showing 1-10 of 86 references
Showing 1-10 of 890 extracted citations
0100200300200920102011201220132014201520162017
Citations per Year

1,716 Citations

Semantic Scholar estimates that this publication has received between 1,517 and 1,940 citations based on the available data.

See our FAQ for additional information.