On Model Coding for Distributed Inference and Transmission in Mobile Edge Computing Systems

@article{Zhang2019OnMC,
  title={On Model Coding for Distributed Inference and Transmission in Mobile Edge Computing Systems},
  author={Jingjing Zhang and Osvaldo Simeone},
  journal={IEEE Communications Letters},
  year={2019},
  volume={23},
  pages={1065-1068}
}
Consider a mobile edge computing system in which users wish to obtain the result of a linear inference operation on locally measured input data. Unlike the offloaded input data, the model weight matrix is distributed across the wireless Edge Nodes (ENs). ENs have non-deterministic computing times, and they can transmit any shared computed output back to the users cooperatively. This letter investigates the potential advantages obtained by coding model information prior to ENs’ storage. Through… 

Figures from this paper

Rateless Codes for Low-Latency Distributed Inference in Mobile Edge Computing
TLDR
This work considers a mobile edge computing scenario where users want to perform a linear inference operation Wx on local data x for some network-side matrix W and proposes a coding scheme that combines a rateless code to provide resiliency against straggling servers and an irregular-repetition code to provides spatial diversity—hence reducing the communication latency.
Multi-Cell Mobile Edge Coded Computing: Trading Communication and Computing for Distributed Matrix Multiplication
TLDR
A policy based on cascaded coded computing and on coordinated and cooperative interference management in uplink and downlink is proposed and proved to be approximately optimal for sufficiently large upload times.
Coding for Mobile Edge Computing Using Luby-Transform codes to lower the latency in edge computing systems
TLDR
In this thesis two coding schemes inspired by the work of Zhang et al. are proposed and evaluated by simulations, indicating a lower total latency for one of the proposed schemes as compared to the MDS hybrid.
Optimal Load Allocation for Coded Distributed Computation in Heterogeneous Clusters
TLDR
This article focuses on the scenario that there exist workers having the same computing capability, which can be regarded as a group for analysis and proposes the optimal load allocation for coded distributed computing with heterogeneous workers.
Communication-Efficient Edge AI: Algorithms and Systems
TLDR
A comprehensive survey of the recent developments in various techniques for overcoming key communication challenges in edge AI systems is presented, and communication-efficient techniques are introduced from both algorithmic and system perspectives for training and inference tasks at the network edge.
Incentive-Based Coded Distributed Computing Management for Latency Reduction in IoT Services—A Game Theoretic Approach
TLDR
Novel coded DC mechanisms with two different incentive distributions that consider the time-discounting value of processed results and the amount of the workload computed by MDs for latency reduction in Internet-of-Things services by mitigating the effect of straggling MDs are proposed.
Privacy-Preserving Coded Mobile Edge Computing for Low-Latency Distributed Inference
TLDR
A coding scheme that provides information-theoretic privacy against colluding (honest-but-curious) edge servers, while minimizing the overall latency—comprising upload, computation, download, and decoding latency—in the presence of straggling servers is proposed.
Coding for Straggler Mitigation in Federated Learning
We present a novel coded federated learning (FL) scheme for linear regression that mitigates the effect of straggling devices while retaining the privacy level of conventional FL. The proposed scheme
Computational Code-Based Privacy in Coded Federated Learning
TLDR
A privacy-preserving federated learning (FL) scheme that is resilient against straggling devices and employs code-based cryptography to ensure computational privacy of the private data is proposed.
CodedPaddedFL and CodedSecAgg: Straggler Mitigation and Secure Aggregation in Federated Learning
TLDR
Two novel coded federated learning (FL) schemes for linear regression that mitigate the effect of straggling devices and provide straggler resiliency and robustness against model inversion attacks are presented.
...
1
2
...

References

SHOWING 1-10 OF 14 REFERENCES
Fundamental Limits of Cloud and Cache-Aided Interference Management With Multi-Antenna Edge Nodes
TLDR
The analysis characterizes the minimum high-SNR latency in terms of normalized delivery time (NDT) for worst case users’ demands and brings insights into the optimal interplay between edge and cloud processing in fog-aided wireless networks as a function of system resources.
A Unified Coding Framework for Distributed Computing with Straggling Servers
TLDR
An information-theoretic lower bound on the latency- load tradeoff is proved, which is shown to be within a constant multiplicative gap from the achieved tradeoff at the two end points.
Coding for Distributed Fog Computing
TLDR
This work discusses two recently proposed coding concepts, minimum bandwidth codes and minimum latency codes, and illustrates their impacts on fog computing, and reviews a unified coding framework that includes the above two coding techniques as special cases.
Joint Optimization of Radio and Computational Resources for Multicell Mobile-Edge Computing
TLDR
This paper considers an MIMO multicell system where multiple mobile users ask for computation offloading to a common cloud server, and proposes an iterative algorithm, based on a novel successive convex approximation technique, converging to a local optimal solution of the original nonconvex problem.
Fundamental limits of cache-aided interference management
TLDR
A system comprising a library of files and a wireless network with an arbitrary number of transmitters and receivers, where each node is equipped with a local cache memory, demonstrates that in this setting, caches at the transmitters' side are equally valuable as the caches on the receivers' side, and shows that caching can offer a throughput gain that scales linearly with the size of the network.
Distributed Gradient Descent with Coded Partial Gradient Computations
TLDR
A hybrid approach is introduced, called coded partial gradient computation (CPGC), that benefits from the advantages of both coded and uncoded computation schemes, and reduces both the computation time and decoding complexity.
Rateless Codes for Near-Perfect Load Balancing in Distributed Matrix-Vector Multiplication
TLDR
This paper proposes a rateless fountain coding strategy that achieves the best of both worlds -- it is proved that its latency is asymptotically equal to ideal load balancing, and it performs asymPTotically zero redundant computations.
Improved Latency-communication Trade-off for Map-shuffle-reduce Systems with Stragglers
  • Jingjing Zhang, O. Simeone
  • Computer Science
    ICASSP 2019 - 2019 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)
  • 2019
TLDR
It is demonstrated that it is possible to leverage the redundancy created by repetition coding in order to increased the rate of the outer MDS code and hence to increase the multicasting opportunities in the shuffle phase.
On Multi-Access Edge Computing: A Survey of the Emerging 5G Network Edge Cloud Architecture and Orchestration
TLDR
This paper analyzes the MEC reference architecture and main deployment scenarios, which offer multi-tenancy support for application developers, content providers, and third parties, and elaborates further on open research challenges.
Speeding Up Distributed Machine Learning Using Codes
TLDR
This paper focuses on two of the most basic building blocks of distributed learning algorithms: matrix multiplication and data shuffling, and uses codes to reduce communication bottlenecks, exploiting the excess in storage.
...
1
2
...