Finding Structure with Randomness: Probabilistic Algorithms for Constructing Approximate Matrix Decompositions
@article{Halko2011FindingSW, title={Finding Structure with Randomness: Probabilistic Algorithms for Constructing Approximate Matrix Decompositions}, author={Nathan Halko and Per-Gunnar Martinsson and Joel A. Tropp}, journal={SIAM Rev.}, year={2011}, volume={53}, pages={217-288} }
Low-rank matrix approximations, such as the truncated singular value decomposition and the rank-revealing QR decomposition, play a central role in data analysis and scientific computing. This work surveys and extends recent research which demonstrates that randomization offers a powerful tool for performing low-rank matrix approximation. These techniques exploit modern computational architectures more fully than classical methods and open the possibility of dealing with truly massive data sets…
Figures and Tables from this paper
3,154 Citations
The PowerURV algorithm for computing rank-revealing full factorizations
- Computer ScienceArXiv
- 2018
This work introduces a new randomized algorithm for producing rank-revealing factorizations based on existing work by Demmel, Dumitriu and Holtz that results in close-to optimal low-rank approximations to a given matrix.
Compressing Rank-Structured Matrices via Randomized Sampling
- Computer ScienceSIAM J. Sci. Comput.
- 2016
The proposed scheme is also useful in simplifying the implementation of certain operations on rank-structured matrices such as the matrix-matrix multiplication, low-rank update, addition, etc.
Random Projections and Dimension Reduction
- Computer ScienceArXiv
- 2020
It is described how randomization can be used to create more efficient algorithms to perform low-rank matrix approximation, as well as introducing a novel randomized algorithm for matrix decomposition.
Randomized Matrix Decompositions using
- Computer Science
- 2017
This work presents the R package rsvd, and provides a tutorial introduction to randomized matrix decompositions, showing the computational advantage over other methods implemented in R for approximating matrices with low-rank structure.
Randomized Matrix Decompositions using R
- Computer ScienceJournal of Statistical Software
- 2019
This work presents the R package rsvd, and provides a tutorial introduction to randomized matrix decompositions, showing the computational advantage over other methods implemented in R for approximating matrices with low-rank structure.
Randomized methods for computing low-rank approximations of matrices
- Computer Science
- 2012
The dissertation describes a set of randomized techniques for rapidly constructing a low-rank approximation to a matrix and presents a parallelized randomized scheme for computing a reduced rank Singular Value Decomposition.
Randomized Projection for Rank-Revealing Matrix Factorizations and Low-Rank Approximations
- Computer ScienceSIAM Rev.
- 2020
The truncated version of RQRCP provides a key initial step in the authors' truncated SVD approximation, TUXV, and opens up a new performance domain for large matrix factorizations that will support efficient problem-solving techniques for challenging applications in science, engineering, and data analysis.
Low Rank Approximation and Decomposition of Large Matrices Using Error Correcting Codes
- Computer ScienceIEEE Transactions on Information Theory
- 2017
It is shown how matrices from error correcting codes can be used to find such low rank approximations and matrix decompositions, and extended the framework to linear least squares regression problems.
Fast randomized numerical rank estimation
- Computer Science, MathematicsArXiv
- 2021
A randomized algorithm for estimating the numerical rank of a matrix based on sketching the matrix with random matrices from both left and right; the key fact is that with high probability, the sketches preserve the orders of magnitude of the leading singular values.
High Dimensional Low Rank Plus Sparse Matrix Decomposition
- Computer ScienceIEEE Transactions on Signal Processing
- 2017
A scalable subspace-pursuit approach that transforms the decomposition problem to a subspace learning problem is proposed and adaptive sampling makes the required number of sampled columns/rows invariant to the distribution of the data.
References
SHOWING 1-10 OF 197 REFERENCES
Error Bounds for Random Matrix Approximation Schemes
- Computer Science
- 2009
The spectral error bound is used to predict the performance of several sparsification and quantization schemes that have appeared in the literature; the results are competitive with the performance guarantees given by earlier scheme-specific analyses.
Relative-Error CUR Matrix Decompositions
- Computer Science, MathematicsSIAM J. Matrix Anal. Appl.
- 2008
These two algorithms are the first polynomial time algorithms for such low-rank matrix approximations that come with relative-error guarantees; previously, in some cases, it was not even known whether such matrix decompositions exist.
Improved Approximation Algorithms for Large Matrices via Random Projections
- Computer Science2006 47th Annual IEEE Symposium on Foundations of Computer Science (FOCS'06)
- 2006
The key idea is that low dimensional embeddings can be used to eliminate data dependence and provide more versatile, linear time pass efficient matrix computation.
Exact Matrix Completion via Convex Optimization
- Computer Science, MathematicsFound. Comput. Math.
- 2009
It is proved that one can perfectly recover most low-rank matrices from what appears to be an incomplete set of entries, and that objects other than signals and images can be perfectly reconstructed from very limited information.
Fast Monte-Carlo algorithms for finding low-rank approximations
- Computer ScienceProceedings 39th Annual Symposium on Foundations of Computer Science (Cat. No.98CB36280)
- 1998
This paper develops an algorithm which is qualitatively faster provided the entries of the matrix are sampled according to a natural probability distribution and the algorithm takes time polynomial in k, 1//spl epsiv/, log(1//spl delta/) only, independent of m, n.
Numerical linear algebra in the streaming model
- Computer ScienceSTOC '09
- 2009
Near-optimal space bounds are given in the streaming model for linear algebra problems that include estimation of matrix products, linear regression, low-rank approximation, and approximation of matrix rank; results for turnstile updates are proved.
A fast randomized algorithm for the approximation of matrices ✩
- Mathematics, Computer Science
- 2007
Fast Monte Carlo Algorithms for Matrices II: Computing a Low-Rank Approximation to a Matrix
- Computer ScienceSIAM J. Comput.
- 2006
Two simple and intuitive algorithms are presented which compute a description of a low-rank approximation of a singular value decomposition (SVD) to an matrix of rank not greater than a specified rank, and which are qualitatively faster than the SVD.
Using randomization to make recursive matrix algorithms practical
- Computer ScienceJournal of Functional Programming
- 1999
This work shows how to randomize an input matrix to guarantee that submatrices of the input matrix meet these requirements, and to make recursive block decomposition methods practical on well-conditioned input matrices.