• Corpus ID: 208157929

Can You Really Backdoor Federated Learning?

@article{Sun2019CanYR,
  title={Can You Really Backdoor Federated Learning?},
  author={Ziteng Sun and Peter Kairouz and Ananda Theertha Suresh and H. B. McMahan},
  journal={ArXiv},
  year={2019},
  volume={abs/1911.07963}
}
The decentralized nature of federated learning makes detecting and defending against adversarial attacks a challenging task. This paper focuses on backdoor attacks in the federated learning setting, where the goal of the adversary is to reduce the performance of the model on targeted tasks while maintaining good performance on the main task. Unlike existing works, we allow non-malicious clients to have correctly labeled samples from the targeted tasks. We conduct a comprehensive study of… 
[PDF] Semantic Reader
279 Citations
Highly Influential Citations
49
Background Citations
171
Methods Citations
60
Results Citations
2

Figures from this paper

279 Citations

Towards Mitigating Poisoning Attacks in Federated Learning

This paper proposes ARMOR, a novel mechanism that successfully detects backdoor attacks in Federated Learning and describes the design principles of ARMOR based on generative adversarial networks, which demonstrates that it outperforms its competitors.

Dynamic backdoor attacks against federated learning

This paper bridges meta-learning and backdoor attacks under FL setting, in which case the algorithm can learn a versatile model from previous experiences, and fast adapting to new adversarial tasks with a few of examples.

Backdoor Attacks and Defenses in Federated Learning: State-of-the-Art, Taxonomy, and Future Directions

This work presents a comprehensive review of the state-of-the-art backdoor attacks and defenses in federated learning, and classifies the existing backdoor attacks into two categories: data poisoning attacks and model poisoning attacks, and divides the defenses into anomaly updates detection, robust federated training, and backdoored model restoration.

FedGrad: Mitigating Backdoor Attacks in Federated Learning Through Local Ultimate Gradients Inspection

FedGrad is proposed, a novel backdoor-resistant defense for FL that is resistant to cutting-edge backdoor attacks, including the edge-case attack, and performs effectively under heterogeneous client data and a large number of compromised clients.

Attack of the Tails: Yes, You Really Can Backdoor Federated Learning

Evidence is provided that, in the general case, robustness to backdoors implies model robusts to adversarial examples, and that detecting the presence of a backdoor in a FL model is unlikely assuming first order oracles or polynomial time.

Backdoor Attacks and Defenses in Federated Learning: Survey, Challenges and Future Research Directions

This survey provides a comprehensive survey of current backdoor attack strategies and defenses in FL, including a comprehensive analysis of different approaches.

Meta Federated Learning

The proposed Meta Federated Learning (Meta-FL), a novel variant of federated learning which not only is compatible with secure aggregation protocol but also facilitates defense against backdoor attacks, is proposed.

Towards Mitigation of Edge-Case Backdoor Attacks in Federated Learning

ARMOR is the first FL defense mechanism against targeted poisoning attacks that is compatible with secure aggregation, thus providing better privacy than its competitors.

Defending Against Backdoors in Federated Learning with Robust Learning Rate

This work first conjecture the necessary steps to carry a successful backdoor attack in FL setting, and then, explicitly formulate the defense based on this conjecture, and provides empirical evidence that supports it, and test the defense against backdoor attacks under different settings.

Sniper Backdoor: Single Client Targeted Backdoor Attack in Federated Learning

This paper introduces a novel technique to allow backdoor attacks to be client-targeted, compromising a single client while the rest remain unchanged, and takes advantage of state-of-the-art model inversion and backdoor attacks.
...

32 References

How To Backdoor Federated Learning

This work designs and evaluates a new model-poisoning methodology based on model replacement and demonstrates that any participant in federated learning can introduce hidden backdoor functionality into the joint global model, e.g., to ensure that an image classifier assigns an attacker-chosen label to images with certain features.

Analyzing Federated Learning through an Adversarial Lens

This work explores the threat of model poisoning attacks on federated learning initiated by a single, non-colluding malicious agent where the adversarial objective is to cause the model to misclassify a set of chosen inputs with high confidence.

Targeted Backdoor Attacks on Deep Learning Systems Using Data Poisoning

This work considers a new type of attacks, called backdoor attacks, where the attacker's goal is to create a backdoor into a learning-based authentication system, so that he can easily circumvent the system by leveraging the backdoor.

Fine-Pruning: Defending Against Backdooring Attacks on Deep Neural Networks

Fine-pruning is evaluated, a combination of pruning and fine-tuning, and it is shown that it successfully weakens or even eliminates the backdoors, i.e., in some cases reducing the attack success rate to 0% with only a \(0.4\%\) drop in accuracy for clean (non-triggering) inputs.

Support vector machines under adversarial label contamination

Backdoor Embedding in Convolutional Neural Network Models via Invisible Perturbation

This paper proposes two approaches for generating a backdoor that is hardly perceptible yet effective in poisoning the model, and demonstrates that such attacks can be effective and achieve a high attack success rate at a small cost of model accuracy loss with a small injection rate.

Certified Defenses for Data Poisoning Attacks

This work addresses the worst-case loss of a defense in the face of a determined attacker by constructing approximate upper bounds on the loss across a broad family of attacks, for defenders that first perform outlier removal followed by empirical risk minimization.

The Hidden Vulnerability of Distributed Learning in Byzantium

Bulyan is introduced, and it is empirically show that Bulyan does not suffer the fragility of existing aggregation rules and, at a reasonable cost in terms of required batch size, achieves convergence as if only non-Byzantine gradients had been used to update the model.

BadNets: Evaluating Backdooring Attacks on Deep Neural Networks

It is shown that the outsourced training introduces new security risks: an adversary can create a maliciously trained network (a backdoored neural network, or a BadNet) that has the state-of-the-art performance on the user's training and validation samples but behaves badly on specific attacker-chosen inputs.

Machine Learning with Adversaries: Byzantine Tolerant Gradient Descent

Krum is proposed, an aggregation rule that satisfies the resilience property of the aggregation rule capturing the basic requirements to guarantee convergence despite f Byzantine workers, which is argued to be the first provably Byzantine-resilient algorithm for distributed SGD.