• Corpus ID: 49557410

How To Backdoor Federated Learning

@article{Bagdasaryan2018HowTB,
  title={How To Backdoor Federated Learning},
  author={Eugene Bagdasaryan and Andreas Veit and Yiqing Hua and Deborah Estrin and Vitaly Shmatikov},
  journal={ArXiv},
  year={2018},
  volume={abs/1807.00459}
}
Federated learning enables multiple participants to jointly construct a deep learning model without sharing their private training data with each other. [] Key Result We also show how to evade anomaly detection-based defenses by incorporating the evasion into the loss function when training the attack model.
[PDF] Semantic Reader
972 Citations
Highly Influential Citations
165
Background Citations
675
Methods Citations
190
Results Citations
15

972 Citations

Backdoor Attacks on Federated Meta-Learning

This paper proposes a defense mechanism inspired by matching networks, where the class of an input is predicted from the cosine similarity of its features with a support set of labeled examples, and removes the decision logic from the model shared with the federation.

Defending against Poisoning Backdoor Attacks on Federated Meta-learning

This article proposes a defense mechanism inspired by matching networks, where the class of an input is predicted from the similarity of its features with a support set of labeled examples, and removes the decision logic from the model shared with the federation.

A Federated Learning Backdoor Attack Defense

A federated learning backdoor attack defense based on dual attention mechanism (FDDAM) is proposed, where the model weights are dynamically adjusted during training process, no additional models are required, and the calculation time is shorter.

DeepSight: Mitigating Backdoor Attacks in Federated Learning Through Deep Model Inspection

The performance and effectiveness of DeepSight is evaluated and it is shown that it can mitigate state-of-the-art backdoor attacks with a negligible impact on the model’s performance on benign data.

On the Vulnerability of Backdoor Defenses for Federated Learning

This paper studies whether the current defense mechanisms truly neutralize the backdoor threats from federated learning in a practical setting by proposing a new federated backdoor attack method for possible countermeasures and provides suggestions to the practitioners when training federated models in practice.

Toward Cleansing Backdoored Neural Networks in Federated Learning

A new and effective method to mitigate backdoor attacks in federated learning after the training phase is proposed, which removes redundant neurons and "backdoor neurons", which trigger misbehavior upon recognizing backdoor patterns while keeping silent when the input data is clean.

Mitigating Backdoor Attacks in Federated Learning

A federated pruning method to remove redundant neurons in the network and then adjust the model's extreme weight values is designed, which can reduce the average attack success rate and minimize the pruning influence on test accuracy.

FLAME: Taming Backdoors in Federated Learning

Evaluation of FLAME on several datasets stemming from application areas including image classification, word prediction, and IoT intrusion detection demonstrates that FLAME removes backdoors effectively with a negligible impact on the benign performance of the models.

Dynamic backdoor attacks against federated learning

This paper bridges meta-learning and backdoor attacks under FL setting, in which case the algorithm can learn a versatile model from previous experiences, and fast adapting to new adversarial tasks with a few of examples.

Backdoor Attacks and Defenses in Federated Learning: State-of-the-Art, Taxonomy, and Future Directions

This work presents a comprehensive review of the state-of-the-art backdoor attacks and defenses in federated learning, and classifies the existing backdoor attacks into two categories: data poisoning attacks and model poisoning attacks, and divides the defenses into anomaly updates detection, robust federated training, and backdoored model restoration.
...

80 References

Mitigating Sybils in Federated Learning Poisoning

FoolsGold is described, a novel defense to this problem that identifies poisoning sybils based on the diversity of client updates in the distributed learning process that exceeds the capabilities of existing state of the art approaches to countering sybil-based label-flipping and backdoor poisoning attacks.

Analyzing Federated Learning through an Adversarial Lens

This work explores the threat of model poisoning attacks on federated learning initiated by a single, non-colluding malicious agent where the adversarial objective is to cause the model to misclassify a set of chosen inputs with high confidence.

Exploiting Unintended Feature Leakage in Collaborative Learning

This work shows that an adversarial participant can infer the presence of exact data points -- for example, specific locations -- in others' training data and develops passive and active inference attacks to exploit this leakage.

BadNets: Identifying Vulnerabilities in the Machine Learning Model Supply Chain

It is shown that outsourced training introduces new security risks: an adversary can create a maliciously trained network (a backdoored neural network, or a BadNet) that has state-of-the-art performance on the user's training and validation samples, but behaves badly on specific attacker-chosen inputs.

Targeted Backdoor Attacks on Deep Learning Systems Using Data Poisoning

This work considers a new type of attacks, called backdoor attacks, where the attacker's goal is to create a backdoor into a learning-based authentication system, so that he can easily circumvent the system by leveraging the backdoor.

Detecting Backdoor Attacks on Deep Neural Networks by Activation Clustering

This work proposes a novel approach to backdoor detection and removal for neural networks that is the first methodology capable of detecting poisonous data crafted to insert backdoors and repairing the model that does not require a verified and trusted dataset.

Comprehensive Privacy Analysis of Deep Learning: Stand-alone and Federated Learning under Passive and Active White-box Inference Attacks

It is shown that even well-generalized models are significantly susceptible to white-box membership inference attacks, by analyzing state-of-the-art pre-trained and publicly available models for the CIFAR dataset.

Auror: defending against poisoning attacks in collaborative deep learning systems

This paper investigates the setting of indirect collaborative deep learning --- a form of practical deep learning wherein users submit masked features rather than direct data, and proposes Auror, a system that detects malicious users and generates an accurate model.

Bypassing Backdoor Detection Algorithms in Deep Learning

    T. TanR. Shokri
    Computer Science
    2020 IEEE European Symposium on Security and Privacy (EuroS&P)
  • 2020
An adversarial backdoor embedding algorithm that can bypass the existing detection algorithms including the state-of-the-art techniques, and an adaptive adversarial training algorithm that optimizes the original loss function of the model, and maximizes the indistinguishability of the hidden representations of poisoned data and clean data.

A Little Is Enough: Circumventing Defenses For Distributed Learning

It is shown that 20% of corrupt workers are sufficient to degrade a CIFAR10 model accuracy by 50%, as well as to introduce backdoors into MNIST and CIFar10 models without hurting their accuracy.
...