SAFA: A Semi-Asynchronous Protocol for Fast Federated Learning With Low Overhead

@article{Wu2021SAFAAS,
  title={SAFA: A Semi-Asynchronous Protocol for Fast Federated Learning With Low Overhead},
  author={Wentai Wu and Ligang He and Weiwei Lin and Rui Mao and C. Maple and S. Jarvis},
  journal={IEEE Transactions on Computers},
  year={2021},
  volume={70},
  pages={655-668}
}
Federated learning (FL) has attracted increasing attention as a promising approach to driving a vast number of end devices with artificial intelligence. However, it is very challenging to guarantee the efficiency of FL considering the unreliable nature of end devices while the cost of device-server communication cannot be neglected. In this article, we propose SAFA, a semi-asynchronous FL protocol, to address the problems in federated learning such as low round efficiency and poor convergence… Expand
FedProf: Optimizing Federated Learning with Dynamic Data Profiling
TLDR
This work proposes a novel approach to optimizing FL under such circumstances without breaching data privacy by adapting each client’s opportunity of participation in each FL round to mitigate the impact from the clients with low-quality data. Expand
RC-SSFL: Towards Robust and Communication-efficient Semi-supervised Federated Learning System
TLDR
RC-SSFL employs a minimax optimization-based client selection strategy to select the clients who hold high-quality updates and uses geometric median aggregation to robustly aggregate model updates and implements a novel symmetric quantization method to greatly improve communication efficiency. Expand
Offloading Federated Learning Task to Edge Computing with Trust Execution Environment
TLDR
Considering the time-varying processing capabilities on the FL clients, an exploration-exploitation based client selection algorithm is proposed and it is shown that the proposed algorithm indeed accelerate training process thanks to its efficient client selection. Expand
ELFISH: Resource-Aware Federated Learning on Heterogeneous Edge Devices
TLDR
ELFISH improves the model accuracy for 4% with even better collaborative convergence robustness and overcomes the computational heterogeneity of edge devices and achieves synchronized collaboration without computational stragglers. Expand
A Survey on federated learning*
  • L. Li, Yuxi Fan, Kuo-Yi Lin
  • Computer Science
  • 2020 IEEE 16th International Conference on Control & Automation (ICCA)
  • 2020
TLDR
This study aims to review related studies of FL to base on the baseline a universal definition gives a guiding for the future work and identifies four research fronts to enrich the FL literature and help advance the understanding of the field. Expand
Client Selection for Federated Learning With Non-IID Data in Mobile Edge Computing
TLDR
This paper proposes an efficient FL algorithm, named CSFedAvg, in which the clients with lower degree of non-IID data will be chosen to train the models with higher frequency, and conducts simulations using publicly-available datasets to train deep neural networks. Expand
Analysis of Federated Learning as a Distributed Solution for Learning on Edge Devices
TLDR
The results confirm that it is possible to conduct a complex learning task on distributed devices without sharing their data with a central server, and such federated models can achieve comparable accuracy to central models. Expand
An Efficiency-Boosting Client Selection Scheme for Federated Learning With Fairness Guarantee
TLDR
An estimation of the model exchange time between each client and the server is proposed, based on which a fairness guaranteed algorithm termed RBCS-F for problem-solving is designed. Expand
Server Averaging for Federated Learning
TLDR
This work proposes the server averaging algorithm, which not only converges faster, to a target accuracy, than federated averaging (FedAvg), but also reduces the computation costs on the client-level through epoch decay. Expand
Towards On-Device Federated Learning: A Direct Acyclic Graph-based Blockchain Approach
TLDR
This paper introduces a framework for empowering FL using Direct Acyclic Graph (DAG)-based blockchain systematically (D AG-FL), and shows that DAG-FL can achieve better performance in terms of training efficiency and model accuracy compared with the typical on-device federated learning systems as the benchmarks. Expand
...
1
2
3
...

References

SHOWING 1-10 OF 37 REFERENCES
Client Selection for Federated Learning with Heterogeneous Resources in Mobile Edge
  • T. Nishio, Ryo Yonetani
  • Computer Science
  • ICC 2019 - 2019 IEEE International Conference on Communications (ICC)
  • 2019
TLDR
The new FedCS protocol, which the authors refer to as FedCS, solves a client selection problem with resource constraints, which allows the server to aggregate as many client updates as possible and to accelerate performance improvement in ML models. Expand
Federated Learning: Strategies for Improving Communication Efficiency
TLDR
Two ways to reduce the uplink communication costs are proposed: structured updates, where the user directly learns an update from a restricted space parametrized using a smaller number of variables, e.g. either low-rank or a random mask; and sketched updates, which learn a full model update and then compress it using a combination of quantization, random rotations, and subsampling. Expand
Federated Multi-Task Learning
TLDR
This work shows that multi-task learning is naturally suited to handle the statistical challenges of this setting, and proposes a novel systems-aware optimization method, MOCHA, that is robust to practical systems issues. Expand
Adaptive Federated Learning in Resource Constrained Edge Computing Systems
TLDR
This paper analyzes the convergence bound of distributed gradient descent from a theoretical point of view, and proposes a control algorithm that determines the best tradeoff between local update and global parameter aggregation to minimize the loss function under a given resource budget. Expand
Asynchronous Federated Learning for Geospatial Applications
TLDR
This work presents a new asynchronous federated-learning algorithm (‘asynchronous federated learning’) and studies its convergence rate when distributed across many edge devices, with hard data constraints, relative to training the same model on a single device. Expand
When Edge Meets Learning: Adaptive Control for Resource-Constrained Distributed Machine Learning
TLDR
This paper analyzes the convergence rate of distributed gradient descent from a theoretical point of view, and proposes a control algorithm that determines the best trade-off between local update and global parameter aggregation to minimize the loss function under a given resource budget. Expand
Asynchronous Federated Optimization
TLDR
It is proved that the proposed asynchronous federated optimization algorithm has near-linear convergence to a global optimum, for both strongly and non-strongly convex problems, as well as a restricted family of non-convex problems. Expand
Staleness-Aware Async-SGD for Distributed Deep Learning
TLDR
This paper proposes a variant of the ASGD algorithm in which the learning rate is modulated according to the gradient staleness and provides theoretical guarantees for convergence of this algorithm. Expand
Collaborating between Local and Global Learning for Distributed Online Multiple Tasks
TLDR
The experiments from 4 applications show that the proposed method of global learning can improve local learning significantly and is effective, since it can share knowledge among distributed tasks and obtain better models than learning them separately. Expand
Edge Intelligence: On-Demand Deep Learning Model Co-Inference with Device-Edge Synergy
TLDR
Eggent is a collaborative and on-demand DNN co-inference framework with device-edge synergy that pursues two design knobs: DNN partitioning that adaptively partitions DNN computation between device and edge, in order to leverage hybrid computation resources in proximity for real-time DNN inference. Expand
...
1
2
3
4
...