Low rank approximation and regression in input sparsity time

@inproceedings{Clarkson2013LowRA,
  title={Low rank approximation and regression in input sparsity time},
  author={Kenneth L. Clarkson and David P. Woodruff},
  booktitle={STOC '13},
  year={2013}
}
We design a new distribution over poly(r ε<sup>-1</sup>) x n matrices S so that for any fixed n x d matrix A of rank r, with probability at least 9/10, SAx<sub>2</sub> = (1 pm ε)Ax<sub>2</sub> simultaneously for all x ∈ R<sup>d</sup>. Such a matrix S is called a <i>subspace embedding</i>. Furthermore, SA can be computed in O(nnz(A)) + ~O(r<sup>2</sup>ε<sup>-2</sup>) time, where nnz(A) is the number of non-zero entries of A. This improves over all previous subspace embeddings, which required at… 
Low-Rank PSD Approximation in Input-Sparsity Time
TLDR
This work gives algorithms for approximation by low-rank positive semidefinite (PSD) matrices, and shows that there are asymmetric input matrices that cannot have good symmetric column-selected approximations.
OSNAP: Faster Numerical Linear Algebra Algorithms via Sparser Subspace Embeddings
TLDR
The main result is essentially a Bai-Yin type theorem in random matrix theory and is likely to be of independent interest: for any fixed U ∈ R<sup>n×d</sup> with orthonormal columns and random sparse Π, all singular values of ΠU lie in [1 - ε, 1 + ε] with good probability.
Approximation Algorithms for ࡁ0-Low Rank Approximation
TLDR
This work gives the first algorithm with provable guarantees for the l0-Low Rank Approximation Problem for k > 1, even for bicriteria algorithms, and strengthens this for the well-studied case of binary matrices to obtain a (1+O(ψ))-approximation in sublinear time.
Tighter Low-rank Approximation via Sampling the Leveraged Element
TLDR
This work proposes a new randomized algorithm for computing a low-rank approximation to a given matrix that combines the best aspects of otherwise disparate current results, but with a dependence on the condition number κ = σ1/σr.
An homotopy method for lp regression provably beyond self-concordance and in input-sparsity time
TLDR
It is proved that any symmetric self-concordant barrier on the ℓpn unit ball has self- Concordance parameter Ω(n), and a randomized algorithm solving such problems in input sparsity time is proposed, i.e., Õp(N + poly(d)) where N is the size of the input and d is the number of variables.
Input Sparsity Time Low-rank Approximation via Ridge Leverage Score Sampling
We present a new algorithm for finding a near optimal low-rank approximation of a matrix $A$ in $O(nnz(A))$ time. Our method is based on a recursive sampling scheme for computing a representative
D S ] 6 A pr 2 01 8 Tight Bounds for l p Oblivious Subspace Embeddings
TLDR
This paper gives nearly optimal tradeoffs for lp oblivious subspace embeddings with an expected 1+ε number of non-zero entries per column for arbitrarily small ε > 0, and gives the first oblivious sub space embeds for 1 ≤ p < 2 with O(1)-distortion and dimension independent of n.
Empirical Performance of Approximate Algorithms for Low Rank Approximation
TLDR
This data analysis project studies the empirical performance of state-of-the-art in theory approximation algorithms for the `F and `1 variants of the low-rank approximation problem on a variety of synthetic and real datasets.
Optimal CUR matrix decompositions
TLDR
This work presents input-sparsity-time and deterministic algorithms for constructing such a CUR matrix decomposition of A where c = O(k/ε) and r = O (k/ ε) and rank(U) = k and the construction is simultaneously optimal in c, r, andRank(U).
Weighted low rank approximations with provable guarantees
TLDR
The present paper is the first to provide algorithms for the weighted low rank approximation problem with provable guarantees, and this technique turns out to be general enough to give solutions to several other fundamental problems: adversarial matrix completion, weighted non-negative matrix factorization and tensor completion.
...
1
2
3
4
5
...

References

SHOWING 1-10 OF 46 REFERENCES
Low-distortion subspace embeddings in input-sparsity time and applications to robust linear regression
TLDR
These input-sparsity time l<sub>p</sub> embeddings are optimal, up to constants, in terms of their running time; and the improved running time propagates to applications such as (1 pm ε)-distortion l< sub>p </sub> subspace embedding and relative-error l< Sub>p.</sub> regression.
OSNAP: Faster Numerical Linear Algebra Algorithms via Sparser Subspace Embeddings
TLDR
The main result is essentially a Bai-Yin type theorem in random matrix theory and is likely to be of independent interest: for any fixed U ∈ R<sup>n×d</sup> with orthonormal columns and random sparse Π, all singular values of ΠU lie in [1 - ε, 1 + ε] with good probability.
Improved Approximation Algorithms for Large Matrices via Random Projections
  • Tamás Sarlós
  • Computer Science
    2006 47th Annual IEEE Symposium on Foundations of Computer Science (FOCS'06)
  • 2006
TLDR
The key idea is that low dimensional embeddings can be used to eliminate data dependence and provide more versatile, linear time pass efficient matrix computation.
Fast monte-carlo algorithms for finding low-rank approximations
TLDR
An algorithm is developed that is qualitatively faster, provided the authors may sample the entries of the matrix in accordance with a natural probability distribution, and implies that in constant time, it can be determined if a given matrix of arbitrary size has a good low-rank approximation.
Adaptive Sampling and Fast Low-Rank Matrix Approximation
We prove that any real matrix A contains a subset of at most 4k/e+ 2k log(k+1) rows whose span “contains” a matrix of rank at most k with error only (1+e) times the error of the best rank-k
Matrix approximation and projective clustering via volume sampling
TLDR
This paper proves that the additive error drops exponentially by iterating the sampling in an adaptive manner, and gives a pass-efficient algorithm for computing low-rank approximation with reduced additive error.
Sampling from large matrices: An approach through geometric functional analysis
TLDR
The result for the cut-norm yields a slight improvement on the best-known sample complexity for an approximation algorithm for MAX-2CSP problems, and the law of large numbers for operator-valued random variables for Banach spaces is used.
Sparser Johnson-Lindenstrauss Transforms
TLDR
These are the first constructions to provide subconstant sparsity for all values of parameters, improving upon previous works of Achlioptas and Dasgupta et al.
Fast computation of low-rank matrix approximations
TLDR
It is proved that the effect of sampling and quantization nearly vanishes when a low-rank approximation to A + N is computed, which gives high probability bounds on the quality of the approximation both in the Frobenius and the 2-norm.
Subspace Sampling and Relative-Error Matrix Approximation: Column-Based Methods
TLDR
Two polynomial time randomized algorithms that take as input a matrix A and return as output a matrix C sample the columns of A via the method of “subspace sampling,” so-named since the sampling probabilities depend on the lengths of the rows of the top singular vectors and since they ensure that they capture entirely a certain subspace of interest.
...
1
2
3
4
5
...