Cleaning large-dimensional covariance matrices for correlated samples.

  title={Cleaning large-dimensional covariance matrices for correlated samples.},
  author={Zdzislaw Burda and Andrzej Jarosz},
  journal={Physical review. E},
  volume={105 3-1},
  • Z. Burda, A. Jarosz
  • Published 3 July 2021
  • Computer Science, Mathematics
  • Physical review. E
We elucidate the problem of estimating large-dimensional covariance matrices in the presence of correlations between samples. To this end, we generalize the Marčenko-Pastur equation and the Ledoit-Péché shrinkage estimator using methods of random matrix theory and free probability. We develop an efficient algorithm that implements the corresponding analytic formulas based on the Ledoit-Wolf kernel estimation technique. We also provide an associated open-source Python library, called shrinkage… 

Figures and Tables from this paper

Freeness of type $B$ and conditional freeness for random matrices

The asymptotic freeness of independent unitarily invariant N × N random matrices holds in expectation up to O ( N − 2 ). An already known consequence is the infinitesimal freeness in expectation. We

The Cryptocurrency Market in Transition before and after COVID-19: An Opportunity for Investors?

We analyze the correlation between different assets in the cryptocurrency market throughout different phases, specifically bearish and bullish periods. Taking advantage of a fine-grained dataset



A Shrinkage Approach to Large-Scale Covariance Matrix Estimation and Implications for Functional Genomics

This work proposes a novel shrinkage covariance estimator that exploits the Ledoit-Wolf (2003) lemma for analytic calculation of the optimal shrinkage intensity and applies it to the problem of inferring large-scale gene association networks.

Spectrum Estimation: A Unified Framework for Covariance Matrix Estimation and PCA in Large Dimensions

Extensive Monte Carlo simulations demonstrate that the methods used in this paper have desirable finite-sample properties and outperform previous proposals.

Numerical Implementation of the QuEST Function

The present paper explains how to numerically implement the QuEST function in practice through a series of six successive steps and provides an algorithm to compute the Jacobian analytically, which is necessary for numerical inversion by a nonlinear optimizer.

Applying free random variables to random matrix analysis of financial data. Part I: The Gaussian case

We apply the concept of free random variables to doubly correlated (Gaussian) Wishart random matrix models, appearing, for example, in a multivariate analysis of financial time series, and displaying

A random matrix approach to VARMA processes

We apply random matrix theory to derive the spectral density of large sample covariance matrices generated by multivariate VMA(q), VAR(q) and VARMA(q1, q2) processes. In particular, we consider a

Nonlinear Shrinkage of the Covariance Matrix for Portfolio Selection: Markowitz Meets Goldilocks

Markowitz (1952) portfolio selection requires an estimator of the covariance matrix of returns. To address this problem, we promote a nonlinear shrinkage estimator that is more flexible than previous

Free products of large random matrices – a short review of recent developments

We review methods to calculate eigenvalue distributions of products of large random matrices. We discuss a generalization of the law of free multiplication to non-Hermitian matrices and give a couple

Eigenvectors of some large sample covariance matrix ensembles

We consider sample covariance matrices $${S_N=\frac{1}{p}\Sigma_N^{1/2}X_NX_N^* \Sigma_N^{1/2}}$$ where XN is a N ×  p real or complex matrix with i.i.d. entries with finite 12th moment and ΣN is a N