A Unified Treatment of Partial Stragglers and Sparse Matrices in Coded Matrix Computation

  title={A Unified Treatment of Partial Stragglers and Sparse Matrices in Coded Matrix Computation},
  author={Anindya Bijoy Das and Aditya Ramamoorthy},
  journal={2021 IEEE Information Theory Workshop (ITW)},
  • Anindya Bijoy Das, A. Ramamoorthy
  • Published 24 September 2021
  • Computer Science, Mathematics
  • 2021 IEEE Information Theory Workshop (ITW)
The overall execution time of distributed matrix computations is often dominated by slow worker nodes (stragglers) over the clusters. Recently, different coding techniques have been utilized to mitigate the effect of stragglers where worker nodes are assigned the task of processing encoded submatrices of the original matrices. In many machine learning or optimization problems the relevant matrices are often sparse. Several coded computation methods operate with dense linear combinations of the… Expand

Figures and Tables from this paper


C3LES: Codes for Coded Computation that Leverage Stragglers
A fine-grained model is proposed that quantifies the level of non-trivial coding needed to obtain the benefits of coding in matrix-vector computation and allows us to leverage partial computations performed by the straggler nodes. Expand
Coded sparse matrix computation schemes that leverage partial stragglers
  • A. Das, A. Ramamoorthy
  • Computer Science, Mathematics
  • 2021 IEEE International Symposium on Information Theory (ISIT)
  • 2021
This work presents schemes that allow for partial computation by stragglers while imposing constraints on the level of coding that is required in generating the encoded submatrices, which significantly reduces the worker computation time as compared to previous approaches and results in improved numerical stability in the decoding process. Expand
Hierarchical coded matrix multiplication
This paper decomposes the overall matrix multiplication task into a hierarchy of heterogeneously sized subtasks and exploits the work completed by stragglers, rather than ignoring it, even if that amount is much less than that completed by the fastest workers. Expand
Exploitation of Stragglers in Coded Computation
This work introduces a scheme that, in addition to using error correction to distribute mixed jobs across nodes, is also able to exploit the work completed by all nodes, including stragglers, to reduce computation time. Expand
Polynomial Codes: an Optimal Design for High-Dimensional Coded Matrix Multiplication
We consider a large-scale matrix multiplication problem where the computation is carried out using a distributed system with a master node and multiple worker nodes, where each worker can store partsExpand
Straggler-Resistant Distributed Matrix Computation via Coding Theory: Removing a Bottleneck in Large-Scale Data Processing
The current big data era routinely requires the processing of large-scale data on massive distributed computing clusters, which presents several opportunities and advantages over traditional computing paradigms, however, it also presents newer challenges where coding-theoretic ideas have recently had a significant impact. Expand
Universally Decodable Matrices for Distributed Matrix-Vector Multiplication
A class of distributed matrix-vector multiplication schemes that are based on codes in the Rosenbloom-Tsfasman metric and universally decodable matrices are presented that allow us to effectively leverage partial computations performed by stragglers. Expand
Rateless Codes for Near-Perfect Load Balancing in Distributed Matrix-Vector Multiplication
This paper proposes a rateless fountain coding strategy that achieves the best of both worlds -- it is proved that its latency is asymptotically equal to ideal load balancing, and it performs asymPTotically zero redundant computations. Expand
Bivariate Hermitian Polynomial Coding for Efficient Distributed Matrix Multiplication
This work shows how bivariate polynomial coding addresses the issues of storage and computation capacity across workers are heterogeneous and lose completely the work done by the straggling workers. Expand
Coded Sparse Matrix Multiplication
A new coded computation strategy, calledparse code, is developed, which achieves near the optimal recovery threshold, low computation overhead, and linear decoding time, and is implemented and demonstrated over both uncoded and current fastest coded strategies. Expand