# Self-organizing Democratized Learning: Towards Large-scale Distributed Learning Systems

@article{Nguyen2020SelforganizingDL,
title={Self-organizing Democratized Learning: Towards Large-scale Distributed Learning Systems},
author={Minh N. H. Nguyen and Shashi Raj Pandey and Tri Nguyen Dang and Eui-nam Huh and Choong Seon Hong and Nguyen H. Tran and Walid Saad},
journal={IEEE transactions on neural networks and learning systems},
year={2020},
volume={PP}
}
• Published 7 July 2020
• Computer Science
• IEEE transactions on neural networks and learning systems
Emerging cross-device artificial intelligence (AI) applications require a transition from conventional centralized learning systems toward large-scale distributed AI systems that can collaboratively perform complex learning tasks. In this regard, democratized learning (Dem-AI) lays out a holistic philosophy with underlying principles for building large-scale distributed and democratized machine learning systems. The outlined principles are meant to study a generalization in distributed learning…

## Figures from this paper

• Computer Science
IEEE Transactions on Big Data
• 2022
Theoretical analysis and empirical experiments show that the proposed algorithm, PANM, is superior to the P2P FL counterparts, and it achieves better performance than the centralized cluster FL method.
• Computer Science
IEEE Internet of Things Journal
• 2022
The hierarchical learning structure of the proposed edge-assisted Dem-AI mechanism, namely Edge-DemLearn, is shown as a practical framework to empower generalization capability in support of FA and validated as a flexible model training mechanism to build a distributed control and aggregation methodology in regions by leveraging the distributed computing infrastructure.
• Xing WangYijun Wang
• Computer Science
ArXiv
• 2022
The proposed asynchronous hierarchical federated learning schema is used to tolerate heterogeneity of the system and achieve fast convergence, and regularized stochastic gradient descent is performed in workers, so that the instability of asynchronous learning can be alleviated.
• Computer Science
ArXiv
• 2022
Network-aware CE-FL is formulated which aims to adaptively optimize all the network elements via tuning their contribution to the learning process, which turns out to be a non-convex mixed integer problem.
• Computer Science
IEEE Communications Letters
• 2022
This strategy combines exploration of data freshness through a random device selection with exploitation through simplified estimates of device contributions to improve the performance of the trained model both in terms of generalization and personalization.
• Engineering
2022 IEEE Power & Energy Society Innovative Smart Grid Technologies Conference (ISGT)
• 2022
A datadriven democratized control architecture considering two democratization pathways to assist transmission system operators, with a targeted use case of developing online proactive islanding strategies is proposed.
• Computer Science
IEEE Communications Surveys & Tutorials
• 2022
This paper panoramically examines the steps needed to efficiently and reliably deploy and operate next-generation THz wireless systems that will synergistically support a fellowship of communication and sensing services and presents the key THz 6G use cases along with their associated major challenges and open problems.
• Computer Science
Middleware
• 2021
This work proposes a unified approach to decentralization and personalization in federated learning that is based on a directed acyclic graph (DAG) of model updates that enables the evolution of specialized models, which focus on a subset of the data and therefore cover non-IID data better than Federated learning in a centralized or blockchain-based setup.
• Computer Science
Computer Communications and Networks
• 2021
This chapter provides a vision for edge intelligence as a key building block of 6G wireless systems, and identifies the need to provide a sustainable and proactive network design and optimization, by enabling an explainable edge intelligence backed by data-science and theory-based models.
• Computer Science
2021 IEEE 93rd Vehicular Technology Conference (VTC2021-Spring)
• 2021
This paper proposes a resource allocation mechanism adaptive to the environment, which can be an efficient solution for air interface congestion that a V2X network often suffers from and aims at granting a higher chance of transmission to a vehicle with a higher crash risk.

## References

SHOWING 1-10 OF 31 REFERENCES

• Computer Science
ArXiv
• 2017
Fashion-MNIST is intended to serve as a direct drop-in replacement for the original MNIST dataset for benchmarking machine learning algorithms, as it shares the same image size, data format and the structure of training and testing splits.
• Computer Science
Found. Trends Mach. Learn.
• 2021
Motivated by the explosive growth in FL research, this paper discusses recent advances and presents an extensive collection of open problems and challenges.
• Computer Science
IEEE Computational Intelligence Magazine
• 2021
A reference design is presented as a guideline to realize future Dem-AI systems, inspired by various interdisciplinary fields and four underlying mechanisms in the design such as plasticity-stability transition mechanism, self-organizing hierarchical structuring, specialized learning, and generalization are introduced.
• Computer Science
MLSys
• 2020
This work introduces a framework, FedProx, to tackle heterogeneity in federated networks, and provides convergence guarantees for this framework when learning over data from non-identical distributions (statistical heterogeneity), and while adhering to device-level systems constraints by allowing each participating device to perform a variable amount of work.
• Computer Science
ArXiv
• 2018
LEAF is proposed, a modular benchmarking framework for learning in federated settings that includes a suite of open-source federated datasets, a rigorous evaluation framework, and a set of reference implementations, all geared towards capturing the obstacles and intricacies of practical federated environments.
• Computer Science
IEEE Transactions on Mobile Computing
• 2023
A centralized algorithm based on the block coordinate descent method and a decentralized JP-miADMM algorithm for solving the joint resource optimization and hyper-learning rate control problem regarding the energy consumption of mobile devices and overall learning time is designed.
• Computer Science
NeurIPS
• 2020
This work proposes an algorithm for personalized FL (pFedMe) using Moreau envelopes as clients' regularized loss functions, which help decouple personalized model optimization from the global model learning in a bi-level problem stylized for personalizedFL.
• Computer Science
ArXiv
• 2020
Information theoretically, it is proved that the mixture of local and global models can reduce the generalization error and a communication-reduced bilevel optimization method is proposed, which reduces the communication rounds to $O(\sqrt{T})$ and can achieve a convergence rate of $O(1/T)$ with some residual error.
• Computer Science
ArXiv
• 2020
A personalized variant of the well-known Federated Averaging algorithm is studied and its performance is characterized by the closeness of underlying distributions of user data, measured in terms of distribution distances such as Total Variation and 1-Wasserstein metric.
• Computer Science
2019 IEEE Global Communications Conference (GLOBECOM)
• 2019
This work forms a utility maximization problem to tackle the difficulty of maintaining communication efficiency when participating clients implement uncoordinated computation strategy during aggregation of model parameters, and proposes a novel crowdsourcing framework involving a number of participating clients with local training data to leverage FL.