Corpus ID: 237532588

Personalized Federated Learning for Heterogeneous Clients with Clustered Knowledge Transfer

  title={Personalized Federated Learning for Heterogeneous Clients with Clustered Knowledge Transfer},
  author={Yae Jee Cho and Jianyu Wang and Tarun Chiruvolu and Gauri Joshi},
Personalized federated learning (FL) aims to train model(s) that can perform well for individual clients that are highly data and system heterogeneous. Most work in personalized FL, however, assumes using the same model architecture at all clients and increases the communication cost by sending/receiving models. This may not be feasible for realistic scenarios of FL. In practice, clients have highly heterogeneous system-capabilities and limited communication resources. In our work, we propose a… Expand

Figures and Tables from this paper


Adaptive Distillation for Decentralized Learning from Heterogeneous Clients
This paper addresses the problem of decentralized learning to achieve a high-performance global model by asking a group of clients to share local models pre-trained with their own data resources by proposing a new decentralized learning method called Decentralized Learning via Adaptive Distillation (DLAD). Expand
Personalized Federated Learning with Theoretical Guarantees: A Model-Agnostic Meta-Learning Approach
A personalized variant of the well-known Federated Averaging algorithm is studied and its performance is characterized by how this performance is affected by the closeness of underlying distributions of user data, measured in terms of distribution distances such as Total Variation and 1-Wasserstein metric. Expand
FedMD: Heterogenous Federated Learning via Model Distillation
This work uses transfer learning and knowledge distillation to develop a universal framework that enables federated learning when each agent owns not only their private data, but also uniquely designed models. Expand
Think Locally, Act Globally: Federated Learning with Local and Global Representations
A new federated learning algorithm is proposed that jointly learns compact local representations on each device and a global model across all devices, which helps to keep device data private and enable communication-efficient training while retaining performance. Expand
Federated Optimization in Heterogeneous Networks
This work introduces a framework, FedProx, to tackle heterogeneity in federated networks, and provides convergence guarantees for this framework when learning over data from non-identical distributions (statistical heterogeneity), and while adhering to device-level systems constraints by allowing each participating device to perform a variable amount of work. Expand
Federated Learning with Positive and Unlabeled Data
Federated learning with Positive and Unlabeled data (FedPU) is proposed, to minimize the expected risk of multiple negative classes by leveraging the labeled data in other clients and theoretically proves that the proposed FedPU can achieve a generalization bound which is no worse than C √ C times of the fully-supervised model. Expand
On the Convergence of Local Descent Methods in Federated Learning
The obtained convergence rates are the sharpest known to date on the convergence of local decant methods with periodic averaging for solving nonconvex federated optimization in both centralized and networked distributed optimization. Expand
Client Selection in Federated Learning: Convergence Analysis and Power-of-Choice Selection Strategies
This paper presents the first convergence analysis of federated optimization for biased client selection strategies, and quantifies how the selection bias affects convergence speed, and proposes Power-of-Choice, a communication- and computation-efficient client selection framework that can flexibly span the trade-off between convergence speed and solution bias. Expand
Practical One-Shot Federated Learning for Cross-Silo Setting
This paper proposes a practical one-shot federated learning algorithm named FedKT that can be applied to any classification models and can flexibly achieve differential privacy guarantees and can significantly outperform the other state-of-the-art Federated learning algorithms with a single communication round. Expand
Distributed Distillation for On-Device Learning
A distributed distillation algorithm where devices communicate and learn from soft-decision (softmax) outputs, which are inherently architecture-agnostic and scale only with the number of classes is introduced. Expand