Communication-Efficient and Distributed Learning Over Wireless Networks: Principles and Applications

@article{Park2020CommunicationEfficientAD,
  title={Communication-Efficient and Distributed Learning Over Wireless Networks: Principles and Applications},
  author={Jihong Park and Sumudu Samarakoon and Anis Elgabli and Joongheon Kim and Mehdi Bennis and Seong-Lyun Kim and M{\'e}rouane Debbah},
  journal={Proceedings of the IEEE},
  year={2020},
  volume={109},
  pages={796-819}
}
Machine learning (ML) is a promising enabler for the fifth-generation (5G) communication systems and beyond. By imbuing intelligence into the network edge, edge nodes can proactively carry out decision-making and, thereby, react to local environmental changes and disturbances while experiencing zero communication latency. To achieve this goal, it is essential to cater for high ML inference accuracy at scale under the time-varying channel and network dynamics, by continuously exchanging fresh… 
[PDF] Semantic Reader
94 Citations
Highly Influential Citations
1
Background Citations
60
Methods Citations
15

94 Citations

Distributed Learning in Wireless Networks: Recent Progress and Future Challenges

This paper provides a holistic set of guidelines on how to deploy a broad range of distributed learning frameworks over real-world wireless communication networks, including federated learning, federated distillation, distributed inference, and multi-agent reinforcement learning.

Decentralized Edge Learning via Unreliable Device-to-Device Communications

A decentralized edge learning framework over wireless networks via unreliable device-to-device (D2D) links is introduced and an optimization problem to minimize the overall model deviation under a given latency requirement by jointly optimizing the broadcast data rate and bandwidth allocation is developed.

Distributed Intelligence in Wireless Networks

A comprehensive overview of recent advances in distributed intelligence in wireless networks under the umbrella of native-AI wireless networks is conducted, with a focus on the basic concepts ofnative- AI wireless networks, on the AIenabled edge computing, and the design of distributed learning architectures for heterogeneous networks.

Fair and Efficient Distributed Edge Learning with Hybrid Multipath TCP

A hybrid multipath TCP (MPTCP) by combining model-based and deep reinforcement learning (DRL) based MPTCP for DEL that strives to realize quicker iteration of DEL and better fairness (by ameliorating stragglers).

Trends in Intelligent Communication Systems: Review of Standards, Major Research Projects, and Identification of Research Gaps

This review reveals that the telecommunication industry and standardisation bodies have been mostly focused on non-real-time RIC, data analytics at the core and the edge, AI-based network slicing, and vendor inter-operability issues, whereas most recent academic research has focused on real- time RIC.

Federated and Meta learning over Non-Wireless and Wireless Networks: A Tutorial

This tutorial conducts a comprehensive review on FL, meta learning, and federated meta learning (FedMeta) to leverage how FL/meta-learning/FedMeta can be designed, optimized, and evolved over non-wireless and wireless networks.

Communication-Efficient Federated Learning Using Censored Heavy Ball Descent

A censoring-based heavy ball (CHB) method for distributed learning in a server-worker architecture that takes advantage of the HB smoothing to eliminate reporting small changes, and provably achieves a linear convergence rate equivalent to that of the classical HB method for smooth and strongly convex objective functions.

Communication and Energy Efficient Slimmable Federated Learning via Superposition Coding and Successive Decoding

Surprisingly, SlimFL achieves even higher accuracy with lower energy footprints than vanilla FL for poor channels and non-IID data distributions, under which vanilla FL converges slowly.

Adaptive Partial Offloading and Resource Harmonization in Wireless Edge Computing-Assisted IoE Networks

An actor–critic reinforcement learning algorithm is proposed to realize an adaptive offloading decision scheme and optimal wireless resource harmonization between the backhaul and fronthaul and results reveal that the proposed algorithm reliably converges and provides approximately 70%, 55%, 36%, and 11% lower total computation overhead than UE execution, random, MEC execution, and DQN-based schemes.

Multichannel ALOHA Optimization for Federated Learning With Multiple Models

This letter considers that devices are involved in the optimization of more than one model in a FL system, and then proposes an optimum method to allocate wireless resources in a multi-channel ALOHA setup that outperforms uniform and fully-shared channel allocations in terms of convergence time.
...

References

SHOWING 1-10 OF 189 REFERENCES

Wireless Network Intelligence at the Edge

In a first of its kind, this article explores the key building blocks of edge ML, different neural network architectural splits and their inherent tradeoffs, as well as theoretical and technical enablers stemming from a wide range of mathematical disciplines.

Machine Learning at the Edge: A Data-Driven Architecture With Applications to 5G Cellular Networks

It is shown that the prediction accuracy improves when based on machine learning algorithms that rely on the controllers’ view and, consequently, on the spatial correlation introduced by the user mobility, with respect to when the prediction is based only on the local data of each single base station.

One-Bit Over-the-Air Aggregation for Communication-Efficient Federated Edge Learning: Design and Convergence Analysis

A comprehensive analysis of the effects of wireless channel hostilities on the convergence rate of the proposed FEEL scheme is provided, showing that the hostilities slow down the convergence of the learning process by introducing a scaling factor and a bias term into the gradient norm.

Communication-Efficient Edge AI: Algorithms and Systems

A comprehensive survey of the recent developments in various techniques for overcoming key communication challenges in edge AI systems is presented, and communication-efficient techniques are introduced from both algorithmic and system perspectives for training and inference tasks at the network edge.

In-Edge AI: Intelligentizing Mobile Edge Computing, Caching and Communication by Federated Learning

The "In-Edge AI" framework is designed in order to intelligently utilize the collaboration among devices and edge nodes to exchange the learning parameters for a better training and inference of the models, and thus to carry out dynamic system-level optimization and application-level enhancement while reducing the unnecessary system communication load.

Wireless Communications for Collaborative Federated Learning

A novel FL framework is introduced, called collaborative FL, which enables edge devices to implement FL with less reliance on a central controller, and a number of communication techniques are proposed so as to improve CFL performance.

Adaptive Federated Learning in Resource Constrained Edge Computing Systems

This paper analyzes the convergence bound of distributed gradient descent from a theoretical point of view, and proposes a control algorithm that determines the best tradeoff between local update and global parameter aggregation to minimize the loss function under a given resource budget.

Machine Learning Meets Communication Networks: Current Trends and Future Challenges

A detailed account of current research on the application of ML in communication networks and important future research challenges are identified and presented to help stir further research in key areas in this direction.

Federated Learning via Over-the-Air Computation

A novel over-the-air computation based approach for fast global model aggregation via exploring the superposition property of a wireless multiple-access channel and providing a difference-of-convex-functions (DC) representation for the sparse and low-rank function to enhance sparsity and accurately detect the fixed-rank constraint in the procedure of device selection.

Toward Massive, Ultrareliable, and Low-Latency Wireless Communication With Short Packets

Recent advances in information theory are reviewed, which provide the theoretical principles that govern the transmission of short packets and these principles are applied to three exemplary scenarios, thereby illustrating how the Transmission of control information can be optimized when the packets are short.
...