Optimizing Pipelined Computation and Communication for Latency-Constrained Edge Learning

@article{Skatchkovsky2019OptimizingPC,
  title={Optimizing Pipelined Computation and Communication for Latency-Constrained Edge Learning},
  author={Nicolas Skatchkovsky and Osvaldo Simeone},
  journal={IEEE Communications Letters},
  year={2019},
  volume={23},
  pages={1542-1546}
}
Consider a device that is connected to an edge processor via a communication channel. The device holds local data that is to be offloaded to the edge processor so as to train a machine learning model, e.g., for regression or classification. Transmission of the data to the learning processor, as well as training based on stochastic gradient descent (SGD), must be both completed within a time limit. Assuming that communication and computation can be pipelined, this letter investigates the optimal… 

Figures from this paper

Dynamic Resource Allocation for Wireless Edge Machine Learning with Latency And Accuracy Guarantees

An algorithm is devised that jointly allocates radio and computation resources in a dynamic fashion, without requiring prior knowledge of the statistics of the channels, task arrivals, and input data, to explore the optimal tradeoff between the overall system energy consumption and the learning accuracy.

Wireless Edge Machine Learning: Resource Allocation and Trade-Offs

A dynamic resource allocation framework for EML based on stochastic Lyapunov optimization for dynamic training and inference of machine learning tasks at the edge of the wireless network, with the goal of exploring the trade-off between energy, delay and learning accuracy.

Goal-Oriented Communication for Edge Learning Based On the Information Bottleneck

A goal-oriented communication system, based on the combination of IB and stochastic optimization, where the IB principle is used to design the encoder in order to find an optimal balance between representation complexity and relevance of the en-coded data with respect to the goal.

1. Wireless Edge Machine Learning in 5G/6G Networks

A dynamic resource allocation framework to enable wireless edge machine learning, which hinges on stochastic Lyapunov optimization, which enables adaptation to time-varying network conditions, without the need of a priori knowledge of the statistics of random context parameters.

Dynamic Ensemble Inference at the Edge

A dynamic resource allocation algorithm that dynamically selects the data quantization level, the device transmit power, and the ES's CPU scheduling, without any prior knowledge of the statistics of wireless channels and data arrivals is proposed.

Exploiting Diversity Via Importance-Aware User Scheduling for Fast Edge Learning

The proposed importance-aware scheduling method can exploit the two-fold multi-user diversity, namely the diversity in both the multiuser channels and the distributed data samples, which leads to faster model convergence than the conventional scheduling schemes that exploit only a single type of diversity.

On Analog Gradient Descent Learning Over Multiple Access Fading Channels

This work develops a novel Gradient-Based Multiple Access (GBMA) algorithm that can approach the convergence rate of the centralized gradient descent (GD) algorithm in large networks, and establishes a finite-sample bound of the error for both convex and strongly convex loss functions with Lipschitz gradient.

Data-Importance Aware User Scheduling for Communication-Efficient Edge Machine Learning

The proposed importance-aware scheduling can exploit the two-fold multi-user diversity, namely the diversity in both the multiuser channels and the distributed data samples, which leads to faster model convergence than the conventional scheduling schemes that exploit only a single type of diversity.

A Sequential Gradient-Based Multiple Access for Distributed Learning over Fading Channels

  • Tomer SeryKobi Cohen
  • Computer Science
    2019 57th Annual Allerton Conference on Communication, Control, and Computing (Allerton)
  • 2019
A novel Gradient-Based Multiple Access (GBMA) algorithm is developed and it is proved that it can approach the convergence rate of the centralized gradient descent (GD) algorithm in large networks under both convex and strongly convex loss functions with Lipschitz gradient.

A Survey on Sematic Communications for Intelligent Wireless Networks

A detailed survey on the recent technological trends in regard to semantic communications for intelligent wireless networks focuses on the semantic communications architecture including the model, and source and channel coding, and detail the cross-layer interaction, and the various goal-oriented communication applications.

References

SHOWING 1-10 OF 12 REFERENCES

Wireless Network Intelligence at the Edge

In a first of its kind, this article explores the key building blocks of edge ML, different neural network architectural splits and their inherent tradeoffs, as well as theoretical and technical enablers stemming from a wide range of mathematical disciplines.

When Edge Meets Learning: Adaptive Control for Resource-Constrained Distributed Machine Learning

This paper analyzes the convergence rate of distributed gradient descent from a theoretical point of view, and proposes a control algorithm that determines the best trade-off between local update and global parameter aggregation to minimize the loss function under a given resource budget.

Machine Learning at the Wireless Edge: Distributed Stochastic Gradient Descent Over-the-Air

This work introduces an analog DSGD scheme, called A-DSGD, motivated by the additive nature of the wireless MAC, where the workers send their gradient estimates over the MAC through the available channel bandwidth without employing any digital code.

Federated Learning of Deep Networks using Model Averaging

This work presents a practical method for the federated learning of deep networks that proves robust to the unbalanced and non-IID data distributions that naturally arise, and allows high-quality models to be trained in relatively few rounds of communication.

Distributed Deep Neural Networks Over the Cloud, the Edge and End Devices

Compared with the traditional method of offloading raw sensor data to be processed in the cloud, DDNN locally processes most sensor data on end devices while achieving high accuracy and is able to reduce the communication cost by a factor of over 20x.

Adaptive Task Allocation for Mobile Edge Learning

The problem of adaptive task allocation for MEL is considered with the aim to maximize the learning accuracy, while guaranteeing that the total times of data distribution/aggregation over heterogeneous channels, and local computation on heterogeneous nodes, are bounded by a preset duration.

Federated Multi-Task Learning

This work shows that multi-task learning is naturally suited to handle the statistical challenges of this setting, and proposes a novel systems-aware optimization method, MOCHA, that is robust to practical systems issues.

Optimization Methods for Large-Scale Machine Learning

A major theme of this study is that large-scale machine learning represents a distinctive setting in which the stochastic gradient method has traditionally played a central role while conventional gradient-based nonlinear optimization techniques typically falter, leading to a discussion about the next generation of optimization methods for large- scale machine learning.

A Brief Introduction to Machine Learning for Engineers

  • O. Simeone
  • Computer Science
    Found. Trends Signal Process.
  • 2018
This monograph aims at providing an introduction to key concepts, algorithms, and theoretical results in machine learning by building on first principles, while also exposing the reader to more advanced topics with extensive pointers to the literature within a unified notation and mathematical framework.

Sparse spatial autoregressions