• Corpus ID: 220831008

Flower: A Friendly Federated Learning Research Framework

  title={Flower: A Friendly Federated Learning Research Framework},
  author={Daniel J. Beutel and Taner Topal and Akhil Mathur and Xinchi Qiu and Titouan Parcollet and Nicholas D. Lane},
Federated Learning (FL) has emerged as a promising technique for edge devices to collaboratively learn a shared prediction model, while keeping their training data on the device, thereby decoupling the ability to do machine learning from the need to store the data in the cloud. However, FL is difficult to implement and deploy in practice, considering the heterogeneity in mobile devices, e.g., different programming languages, frameworks, and hardware accelerators. Although there are a few… 

FL_PyTorch: optimization research simulator for federated learning

FL_PyTorch is a suite of open-source software written in python that builds on top of one the most popular research Deep Learning (DL) framework PyTorch to enable fast development, prototyping and experimenting with new and existing FL optimization algorithms.

System Optimization in Synchronous Federated Training: A Survey

This paper surveys highly relevant attempts in the FL literature and organizes them by the related training phases in the standard workflow: selection, configuration, and reporting, and reviews exploratory work including measurement studies and benchmarking tools to friendly support FL developers.

FedBN: Federated Learning on Non-IID Features via Local Batch Normalization

This work proposes an effective method that uses local batch normalization to alleviate the feature shift before averaging models, called FedBN, which outperforms both classical FedAvg, as well as the state-of-the-art for non-iid data (FedProx) on the authors' extensive experiments.

On-device Federated Learning with Flower

This paper presents an exploration of on-device FL on various smartphones and embedded devices using the Flower framework and evaluates the system costs and discusses how this quantification could be used to design more efficient FL algorithms.

Leveraging Stack4Things for Federated Learning in Intelligent Cyber Physical Systems

An extension of Stack4Things, a cloud platform developed in this paper, is presented, which enabled the deployment of federated learning on edge devices without caring their heterogeneity and shows the effectiveness of the proposed approach in terms of both training time and model accuracy.

FLAME: Federated Learning Across Multi-device Environments

This paper proposes FLAME, a user-centered FL training approach to counter statistical and system heterogeneity in MDEs, and bring consistency in inference performance across devices.

FedLess: Secure and Scalable Federated Learning Using Serverless Computing

FedLess is the first to enable FL across a large fabric of heterogeneous FaaS providers while providing important features like security and Differential Privacy and demonstrates the practical viability of the methodology by comparing it against a traditional FL system and showing that it can be cheaper and more resource-efficient.

FedLab: A Flexible Federated Learning Framework

The designed and developed FedLab, a flexible and modular FL framework based on PyTorch that provides functional interfaces and a series of baseline implementation are available, making researchers quickly implement ideas.

PySyft: A Library for Easy Federated Learning

This chapter introduces Duet: the authors' tool for easier FL for scientists and data owners and provides a proof-of-concept demonstration of a FL workflow using an example of how to train a convolutional neural network.

Secure aggregation for federated learning in flower

Salvia is an implementation of SA for Python users in the Flower FL framework based on the SecAgg(+) protocols for a semi-honest threat model, which is robust against client dropouts and exposes a flexible and easy-to-use API that is compatible with various machine learning frameworks.



LEAF: A Benchmark for Federated Settings

LEAF is proposed, a modular benchmarking framework for learning in federated settings that includes a suite of open-source federated datasets, a rigorous evaluation framework, and a set of reference implementations, all geared towards capturing the obstacles and intricacies of practical federated environments.

Adaptive Federated Optimization

This work proposes federated versions of adaptive optimizers, including Adagrad, Adam, and Yogi, and analyzes their convergence in the presence of heterogeneous data for general nonconvex settings to highlight the interplay between client heterogeneity and communication efficiency.

Communication-Efficient Learning of Deep Networks from Decentralized Data

This work presents a practical method for the federated learning of deep networks based on iterative model averaging, and conducts an extensive empirical evaluation, considering five different model architectures and four datasets.

TensorFlow: A system for large-scale machine learning

The TensorFlow dataflow model is described and the compelling performance that Tensor Flow achieves for several real-world applications is demonstrated.

Federated Optimization in Heterogeneous Networks

This work introduces a framework, FedProx, to tackle heterogeneity in federated networks, and provides convergence guarantees for this framework when learning over data from non-identical distributions (statistical heterogeneity), and while adhering to device-level systems constraints by allowing each participating device to perform a variable amount of work.

Towards Federated Learning at Scale: System Design

A scalable production system for Federated Learning in the domain of mobile devices, based on TensorFlow is built, describing the resulting high-level design, and sketch some of the challenges and their solutions.

Occlumency: Privacy-preserving Remote Deep-learning Inference Using SGX

This paper designed a suite of novel techniques to accelerate DL inference inside the enclave with a limited memory size and implemented Occlumency based on Caffe, a novel cloud-driven solution designed to protect user privacy without compromising the benefit of using powerful cloud resources.

Horovod: fast and easy distributed deep learning in TensorFlow

Horovod is an open source library that improves on both obstructions to scaling: it employs efficient inter-GPU communication via ring reduction and requires only a few lines of modification to user code, enabling faster, easier distributed training in TensorFlow.

Highly Scalable Deep Learning Training System with Mixed-Precision: Training ImageNet in Four Minutes

This work builds a highly scalable deep learning training system for dense GPU clusters with three main contributions: a mixed-precision training method that significantly improves the training throughput of a single GPU without losing accuracy, an optimization approach for extremely large mini-batch size that can train CNN models on the ImageNet dataset without lost accuracy, and highly optimized all-reduce algorithms.