• Corpus ID: 248427294

Compositional Federated Learning for Distributionally Robust and Meta Learning

@inproceedings{Huang2021CompositionalFL,
  title={Compositional Federated Learning for Distributionally Robust and Meta Learning},
  author={Feihu Huang and Junyi Li},
  year={2021}
}
—In the paper, we propose an effective and efficient Compositional Federated Learning (ComFedL) algorithm for solving a new compositional Federated Learning (FL) framework, which frequently appears in many data mining and machine learning problems with a hierarchical structure such as distributionally robust FL and model-agnostic meta learning (MAML). Moreover, we study the convergence analysis of our ComFedL algorithm under some mild conditions, and prove that it achieves a convergence rate of… 
[PDF] Semantic Reader

Figures from this paper

References

SHOWING 1-10 OF 55 REFERENCES

Personalized Federated Learning: A Meta-Learning Approach

A personalized variant of the well-known Federated Averaging algorithm is studied and its performance is characterized by the closeness of underlying distributions of user data, measured in terms of distribution distances such as Total Variation and 1-Wasserstein metric.

Asynchronous Federated Optimization

It is proved that the proposed asynchronous federated optimization algorithm has near-linear convergence to a global optimum, for both strongly and non-strongly convex problems, as well as a restricted family of non-convex problems.

Distributionally Robust Federated Averaging

This paper is the first to solve distributionally robust federated learning with reduced communication, and to analyze the efficiency of local descent methods on distributed minimax problems.

Trading Redundancy for Communication: Speeding up Distributed SGD for Non-convex Optimization

As the authors move from fully synchronous SGD to distributed local SGD they are trading accuracy with speed, on the other hand, redundancy can increase the accuracy, with roughly the same speed-up, thus they can benefit from the advantages of the two settings in their RI-SGD.

Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networks

We propose an algorithm for meta-learning that is model-agnostic, in the sense that it is compatible with any model trained with gradient descent and applicable to a variety of different learning…

On Bridging Generic and Personalized Federated Learning

This paper proposes a novel federated learning framework that explicitly decouples a model’s dual duties with two prediction tasks, and introduces a family of losses that are robust to non-identical class distributions, enabling clients to train a generic predictor with a consistent objective across them.

Federated Learning Based on Dynamic Regularization

This work proposes a novel federated learning method for distributively training neural network models, where the server orchestrates cooperation between a subset of randomly chosen devices in each round, using a dynamic regularizer for each device at each round.

FLOP: Federated Learning on Medical Datasets using Partial Networks

This work proposes a simple yet effective algorithm, named Federated Learning on Medical Datasets using Partial Networks (FLOP), that shares only a partial model between the server and clients and can allow different hospitals to collaboratively and effectively train a partially shared model without sharing local patients' data.

Ditto: Fair and Robust Federated Learning Through Personalization

This work identifies that robustness to data and model poisoning attacks and fairness, measured as the uniformity of performance across devices, are competing constraints in statistically heterogeneous networks and proposes a simple, general framework, Ditto, that can inherently provide fairness and robustness benefits.

FedCluster: Boosting the Convergence of Federated Learning via Cluster-Cycling

It is shown that FedCluster with the devices implementing the local stochastic gradient descent (SGD) algorithm achieves a faster convergence rate than the conventional federated averaging (Fe) algorithm in the presence of device-level data heterogeneity.
...