Fast linear algebra is stable

@article{Demmel2007FastLA,
  title={Fast linear algebra is stable},
  author={James Demmel and Ioana Dumitriu and Olga Holtz},
  journal={Numerische Mathematik},
  year={2007},
  volume={108},
  pages={59-91}
}
In Demmel et al. (Numer. Math. 106(2), 199–224, 2007) we showed that a large class of fast recursive matrix multiplication algorithms is stable in a normwise sense, and that in fact if multiplication of n-by-n matrices can be done by any algorithm in O(nω+η) operations for any η >  0, then it can be done stably in O(nω+η) operations for any η >  0. Here we extend this result to show that essentially all standard linear algebra operations, including LU decomposition, QR decomposition, linear… 
Fast matrix multiplication is stable
TLDR
It is shown that the exponent of matrix multiplication (the optimal running time) can be achieved by numerically stable algorithms, and new group-theoretic algorithms proposed in Cohn and Umans, and Cohn et al. are all included in the class of algorithms to which the analysis applies.
Minimizing Communication in Numerical Linear Algebra
TLDR
This work generalizes a lower bound on the amount of communication needed to perform dense, n-by-n matrix multiplication using the conventional O(n3) algorithm to a much wider variety of algorithms, including LU factorization, Cholesky factors, LDLT factors, QR factors, the Gram–Schmidt algorithm, and algorithms for eigenvalues and singular values.
Optimal algorithms for linear algebra by quantum inspiration
TLDR
Being derived from quantum intuition, the proposed algorithm is completely disjoint from all previous classical algorithms, and builds on a combination of low-discrepancy sequences and perturbation analysis, and hopes it motivates further exploration of quantum techniques in this respect, hopefully leading to improvements in the understanding of space complexity and numerical stability of these problems.
A Quasi-Random Approach to Matrix Spectral
TLDR
This work develops a completely new, efficient and stable, parallel algorithm to compute an approximate spectral decomposition of any Hermitian matrix, relying on the theory of low-discrepancy or quasi-random sequences a theory which has not been connected thus far to linear algebra problems.
Minimizing Communication in Linear Algebra
TLDR
This work shows how to extend known communication lower bounds for O(n) dense matrix multiplication to all direct linear algebra, i.e. for solving linear systems, least squares problems, eigenproblems and the SVD, for dense or sparse matrices, and for sequential or parallel machines.
Near-Optimal Algorithms for Linear Algebra in the Current Matrix Multiplication Time
TLDR
This work shows how to bypass the main open question regarding the logarithmic factors in the sketching dimension of existing oblivious subspace embeddings that achieve constant-factor approximation, using a refined sketching technique, and obtain optimal or nearly optimal bounds for these problems.
Randomized numerical linear algebra: Foundations and algorithms
TLDR
This survey describes probabilistic algorithms for linear algebraic computations, such as factorizing matrices and solving linear systems, that have a proven track record for real-world problems and treats both the theoretical foundations of the subject and practical computational issues.
A duality relation for matrix pencils with application to linearizations
TLDR
A new class of linearizations is introduced, based on the generalization of a construction used in polynomial algebra to find the zeros of a system of (scalar)Polynomial equations, which is constructed naturally from the QR factorization of the matrix obtained by stacking the coefficients of $A(x)$, has good conditioning and stability properties.
Work-Efficient Matrix Inversion in Polylogarithmic Time
We present an algorithm for inversion of symmetric positive definite matrices that combines the practical requirement of an optimal number of arithmetic operations and the theoretical goal of a
The Quasi-Random Perspective on Matrix Spectral Analysis with Applications
TLDR
This work analyzes the discrepancy of an n-dimensional sequence formed by taking the fractional part of integer multiples of the vector of eigenvalues of the input matrix, and gives rise to a conceptually new algorithm to compute an approximate spectral decomposition of any n x n Hermitian matrix.
...
...

References

SHOWING 1-10 OF 71 REFERENCES
Fast matrix multiplication is stable
TLDR
It is shown that the exponent of matrix multiplication (the optimal running time) can be achieved by numerically stable algorithms, and new group-theoretic algorithms proposed in Cohn and Umans, and Cohn et al. are all included in the class of algorithms to which the analysis applies.
Parallel Algorithm for Solving Some Spectral Problems of Linear Algebra
An inverse free parallel spectral divide and conquer algorithm for nonsymmetric eigenproblems
TLDR
An inverse-free, highly parallel, spectral divide and conquer algorithm that can compute either an invariant subspace of a nonsymmetric matrix, or a pair of left and right deflating subspaces of a regular matrix pencil.
A group-theoretic approach to fast matrix multiplication
  • Henry Cohn, C. Umans
  • Mathematics
    44th Annual IEEE Symposium on Foundations of Computer Science, 2003. Proceedings.
  • 2003
TLDR
A new, group-theoretic approach to bounding the exponent of matrix multiplication is developed, including a proof that certain families of groups of order n/sup 2+o(1)/ support n /spl times/ n matrix multiplication.
Design of a Parallel Nonsymmetric Eigenroutine Toolbox, Part I
TLDR
A toolbox for the dense nonsymmetric eigenproblem is proposed, which includes basic block matrix computations, the matrix sign function, 2-dimensional bisection, and spectral divide and conquer using the matrixSign function to find selected eigenvalues.
Stability of block algorithms with fast level-3 BLAS
TLDR
The numerical stability of the block algorithms in the new linear algebra program library LAPACK is investigated and it is shown that these algorithms have backward error analyses in which the backward error bounds are commensurate with the error bounds for the underlying level-3 BLAS (BLAS3).
Gaussian elimination is not optimal
t. Below we will give an algorithm which computes the coefficients of the product of two square matrices A and B of order n from the coefficients of A and B with tess than 4 . 7 n l°g7 arithmetical
Stability of Parallel Triangular System Solvers
  • N. Higham
  • Mathematics, Computer Science
    SIAM J. Sci. Comput.
  • 1995
TLDR
A forward error bound is identified that holds not only for all the methods described here, but for any triangular equation solver that does not rely on algebraic cancellation; among the implications of the bound is that any such method is extremely accurate for certain special types of triangular systems.
Stability of block LU factorization
TLDR
It is shown here that block LU factorization is stable if A is block diagonally dominant by columns and the level of instability in blockLU factorization can be bounded in terms of the condition number K(A) and the growth factor for Gaussian elimination without pivoting.
...
...