Finding Structure with Randomness: Probabilistic Algorithms for Constructing Approximate Matrix Decompositions

@article{Halko2011FindingSW,
  title={Finding Structure with Randomness: Probabilistic Algorithms for Constructing Approximate Matrix Decompositions},
  author={Nathan Halko and Per-Gunnar Martinsson and Joel A. Tropp},
  journal={SIAM Rev.},
  year={2011},
  volume={53},
  pages={217-288}
}
Low-rank matrix approximations, such as the truncated singular value decomposition and the rank-revealing QR decomposition, play a central role in data analysis and scientific computing. This work surveys and extends recent research which demonstrates that randomization offers a powerful tool for performing low-rank matrix approximation. These techniques exploit modern computational architectures more fully than classical methods and open the possibility of dealing with truly massive data sets… 

Figures and Tables from this paper

The PowerURV algorithm for computing rank-revealing full factorizations
TLDR
This work introduces a new randomized algorithm for producing rank-revealing factorizations based on existing work by Demmel, Dumitriu and Holtz that results in close-to optimal low-rank approximations to a given matrix.
Compressing Rank-Structured Matrices via Randomized Sampling
TLDR
The proposed scheme is also useful in simplifying the implementation of certain operations on rank-structured matrices such as the matrix-matrix multiplication, low-rank update, addition, etc.
Random Projections and Dimension Reduction
TLDR
It is described how randomization can be used to create more efficient algorithms to perform low-rank matrix approximation, as well as introducing a novel randomized algorithm for matrix decomposition.
Randomized Matrix Decompositions using
TLDR
This work presents the R package rsvd, and provides a tutorial introduction to randomized matrix decompositions, showing the computational advantage over other methods implemented in R for approximating matrices with low-rank structure.
Randomized Matrix Decompositions using R
TLDR
This work presents the R package rsvd, and provides a tutorial introduction to randomized matrix decompositions, showing the computational advantage over other methods implemented in R for approximating matrices with low-rank structure.
Randomized methods for computing low-rank approximations of matrices
TLDR
The dissertation describes a set of randomized techniques for rapidly constructing a low-rank approximation to a matrix and presents a parallelized randomized scheme for computing a reduced rank Singular Value Decomposition.
Randomized Projection for Rank-Revealing Matrix Factorizations and Low-Rank Approximations
TLDR
The truncated version of RQRCP provides a key initial step in the authors' truncated SVD approximation, TUXV, and opens up a new performance domain for large matrix factorizations that will support efficient problem-solving techniques for challenging applications in science, engineering, and data analysis.
Low Rank Approximation and Decomposition of Large Matrices Using Error Correcting Codes
TLDR
It is shown how matrices from error correcting codes can be used to find such low rank approximations and matrix decompositions, and extended the framework to linear least squares regression problems.
Fast randomized numerical rank estimation
TLDR
A randomized algorithm for estimating the numerical rank of a matrix based on sketching the matrix with random matrices from both left and right; the key fact is that with high probability, the sketches preserve the orders of magnitude of the leading singular values.
High Dimensional Low Rank Plus Sparse Matrix Decomposition
TLDR
A scalable subspace-pursuit approach that transforms the decomposition problem to a subspace learning problem is proposed and adaptive sampling makes the required number of sampled columns/rows invariant to the distribution of the data.
...
...

References

SHOWING 1-10 OF 197 REFERENCES
Error Bounds for Random Matrix Approximation Schemes
TLDR
The spectral error bound is used to predict the performance of several sparsification and quantization schemes that have appeared in the literature; the results are competitive with the performance guarantees given by earlier scheme-specific analyses.
Relative-Error CUR Matrix Decompositions
TLDR
These two algorithms are the first polynomial time algorithms for such low-rank matrix approximations that come with relative-error guarantees; previously, in some cases, it was not even known whether such matrix decompositions exist.
Improved Approximation Algorithms for Large Matrices via Random Projections
  • Tamás Sarlós
  • Computer Science
    2006 47th Annual IEEE Symposium on Foundations of Computer Science (FOCS'06)
  • 2006
TLDR
The key idea is that low dimensional embeddings can be used to eliminate data dependence and provide more versatile, linear time pass efficient matrix computation.
Exact Matrix Completion via Convex Optimization
TLDR
It is proved that one can perfectly recover most low-rank matrices from what appears to be an incomplete set of entries, and that objects other than signals and images can be perfectly reconstructed from very limited information.
Fast Monte-Carlo algorithms for finding low-rank approximations
  • A. FriezeR. KannanS. Vempala
  • Computer Science
    Proceedings 39th Annual Symposium on Foundations of Computer Science (Cat. No.98CB36280)
  • 1998
TLDR
This paper develops an algorithm which is qualitatively faster provided the entries of the matrix are sampled according to a natural probability distribution and the algorithm takes time polynomial in k, 1//spl epsiv/, log(1//spl delta/) only, independent of m, n.
Numerical linear algebra in the streaming model
TLDR
Near-optimal space bounds are given in the streaming model for linear algebra problems that include estimation of matrix products, linear regression, low-rank approximation, and approximation of matrix rank; results for turnstile updates are proved.
Fast Monte Carlo Algorithms for Matrices II: Computing a Low-Rank Approximation to a Matrix
TLDR
Two simple and intuitive algorithms are presented which compute a description of a low-rank approximation of a singular value decomposition (SVD) to an matrix of rank not greater than a specified rank, and which are qualitatively faster than the SVD.
Using randomization to make recursive matrix algorithms practical
TLDR
This work shows how to randomize an input matrix to guarantee that submatrices of the input matrix meet these requirements, and to make recursive block decomposition methods practical on well-conditioned input matrices.
...
...