• Corpus ID: 246285474

Stochastic diagonal estimation: probabilistic bounds and an improved algorithm

@article{Baston2022StochasticDE,
  title={Stochastic diagonal estimation: probabilistic bounds and an improved algorithm},
  author={Robert A. Baston and Yuji Nakatsukasa},
  journal={ArXiv},
  year={2022},
  volume={abs/2201.10684}
}

• R. Baston, Y. Nakatsukasa
• Published 26 January 2022
• Computer Science, Mathematics
• ArXiv

We study the problem of estimating the diagonal of an implicitly given matrix A. For such a matrix we have access to an oracle that allows us to evaluate the matrix vector product Av. For random variable v drawn from an appropriate distribution, this may be used to return an estimate of the diagonal of the matrix A. Whilst results exist for probabilistic guarantees relating to the error of estimates of the trace of A, no such results have yet been derived for the diagonal. We make two contributions in this regard. We analyse the number of queries s required to guarantee that with probability at least 1 − δ the estimates of the relative error of the diagonal entries is at most ε. We extend this analysis to the 2-norm of the difference between the estimate and the diagonal of A. We prove, discuss and experiment with bounds on the number of queries s required to guarantee a probabilistic bound on the estimates of the diagonal by employing Rademacher and Gaussian random variables. Two sufficient upper bounds on the minimum number of query vectors are proved, extending the work of Avron and Toledo [JACM 58(2)8, 2011], and later work of Roosta-Khorasani and Ascher [FoCM 15, 1187-1212, 2015]. We first prove sufficient bounds for diagonal estimation with Gaussian vectors, and then similarly with Rademacher vectors. We find that, generally, there is little difference between the two, with convergence going as O(log(1/δ)/ε) for individual diagonal elements. However for small s, we find that the Rademacher estimator is superior. These results allow us to then extend the ideas of Meyer, Musco, Musco and Woodruff [SOSA, 142-155, 2021], suggesting algorithm Diag++, to speed up the convergence of diagonal estimation from O(1/ε) to O(1/ε) and make it robust to the spectrum of R. A. Baston and Y. Nakatsukasa Mathematical Institute University of Oxford Tel.: [phone] E-mail: [e-mail], E-mail: [e-mail] ar X iv :2 20 1. 10 68 4v 1 [ cs .D S] 2 6 Ja n 20 22 2 Robert A. Baston, Yuji Nakatsukasa any positive semi-definite matrix A. We analyse our algorithm to demonstrate state-of-the-art convergence.