• Corpus ID: 244908187

When the Curious Abandon Honesty: Federated Learning Is Not Private

@article{Boenisch2021WhenTC,
  title={When the Curious Abandon Honesty: Federated Learning Is Not Private},
  author={Franziska Boenisch and Adam Dziedzic and R. Schuster and Ali Shahin Shamsabadi and Ilia Shumailov and Nicolas Papernot},
  journal={ArXiv},
  year={2021},
  volume={abs/2112.02918}
}
In federated learning (FL), data does not leave personal devices when they are jointly training a machine learning model. Instead, these devices share gradients, parameters, or other model updates, with a central party (e.g., a company) coordinating the training. Because data never “leaves” personal devices, FL is presented as privacypreserving. Yet, recently it was shown that this protection is but a thin facade, as even a passive attacker observing gradients can reconstruct data of individual… 
On the Privacy of Decentralized Machine Learning
TLDR
It is demonstrated that, contrary to what is claimed by decentralized learning proposers, decentralized learning does not offer any security advantages over more practical approaches such as federated learning, and tends to degrade users’ privacy by increasing the attack surface.
Decepticons: Corrupted Transformers Breach Privacy in Federated Learning for Language Models
TLDR
This work proposes a novel attack that reveals private user text by deploying malicious parameter vectors, and which succeeds even with mini-batches, multiple users, and long sequences, suggesting that FL on text is far more vulnerable than previously thought.
Recovering Private Text in Federated Learning of Language Models
TLDR
This paper presents a novel attack method FILM 1 for federated learning of language models and shows the feasibility of recovering text from large batch sizes of up to 128 sentences and demonstrates that FILM can work well with several large-scale datasets.
Truth Serum: Poisoning Machine Learning Models to Reveal Their Secrets
TLDR
It is shown that an adversary who can poison a training dataset can cause models trained on this dataset to leak significant private details of training points belonging to other parties, casting doubts on the relevance of cryptographic privacy guarantees in multiparty computation protocols for machine learning, if parties can arbitrarily select their share of training data.
Fishing for User Data in Large-Batch Federated Learning via Gradient Magnification
TLDR
This work introduces a new strategy that dramatically elevates existing attacks to operate on batches of arbitrarily large size, and without architectural modifications, and demonstrates the strategy in challenging large-scale settings.
Symbolic analysis meets federated learning to enhance malware identifier
TLDR
A federated learning system to identify malwares through the behavioural graphs, i.e., system call dependency graphs, based on a deep learning model including a graph autoencoder and a multi-classifier module and trained by a secure learning protocol among clients to preserve the private data against the inference attacks.
FLAME: Federated Learning Across Multi-device Environments
TLDR
This paper proposes FLAME, a user-centered FL training approach to counter statistical and system heterogeneity in MDEs, and bring consistency in inference performance across devices.
Robbing the Fed: Directly Obtaining Private Data in Federated Learning with Modified Models
TLDR
This work introduces a new threat model based on minimal but malicious modifications of the shared model architecture which enable the server to directly obtain a verbatim copy of user data from gradient updates without solving difficult inverse problems.
PrivFairFL: Privacy-Preserving Group Fairness in Federated Learning
Group fairness ensures that the outcome of machine learning (ML) based decision making systems are not biased towards a certain group of people defined by a sensitive attribute such as gender or
SafeNet: Mitigating Data Poisoning Attacks on Private Machine Learning
—Secure multiparty computation (MPC) has been proposed to allow multiple mutually distrustful data owners to jointly train machine learning (ML) models on their combined data. However, the datasets
...
1
2
...

References

SHOWING 1-10 OF 52 REFERENCES
Federated Learning Attacks Revisited: A Critical Discussion of Gaps, Assumptions, and Evaluation Setups
TLDR
A systematic mapping study of attacks against Federated learning, covering 48 relevant papers from 2016 to the third quarter of 2021, and identifies and discusses several fallacies in the evaluation of attacks, which open up questions on the generalizability of the conclusions.
Beyond Inferring Class Representatives: User-Level Privacy Leakage From Federated Learning
TLDR
This paper gives the first attempt to explore user-level privacy leakage against the federated learning by the attack from a malicious server with a framework incorporating GAN with a multi-task discriminator, which simultaneously discriminates category, reality, and client identity of input samples.
Soteria: Provable Defense against Privacy Leakage in Federated Learning from Representation Perspective
TLDR
This work shows that the data representation leakage from gradients is the essential cause of privacy leakage in FL and proposes a defense called Soteria against model inversion attack in FL, which derives a certified robustness guarantee to FL and a convergence guarantee to FedAvg, after applying the defense.
Exploiting Unintended Feature Leakage in Collaborative Learning
TLDR
This work shows that an adversarial participant can infer the presence of exact data points -- for example, specific locations -- in others' training data and develops passive and active inference attacks to exploit this leakage.
Protection Against Reconstruction and Its Applications in Private Federated Learning
In large-scale statistical learning, data collection and model fitting are moving increasingly toward peripheral devices---phones, watches, fitness trackers---away from centralized data collection.
Salvaging Federated Learning by Local Adaptation
TLDR
This work shows that on standard tasks such as next-word prediction, many participants gain no benefit from FL, and shows that differential privacy and robust aggregation make this problem worse by further destroying the accuracy of the federated model for many participants.
Privacy-Preserving Deep Learning: Revisited and Enhanced
TLDR
A privacy-preserving deep learning system in which many learning participants perform neural network-based deep learning over a combined dataset of all, without actually revealing the participants’ local data to a curious server, which makes use of additively homomorphic encryption.
Prochlo: Strong Privacy for Analytics in the Crowd
TLDR
A principled systems architecture---Encode, Shuffle, Analyze (ESA), which extends existing best-practice methods for sensitive-data analytics, by using cryptography and statistical techniques to make explicit how data is elided and reduced in precision, how only common-enough, anonymous data is analyzed, and how this is done for specific, permitted purposes.
Adversarial Initialization - when your network performs the way I want
TLDR
It is demonstrated in this paper how a simple recipe enables a market player to harm or delay the development of a competing product.
Semi-supervised Knowledge Transfer for Deep Learning from Private Training Data
TLDR
Private Aggregation of Teacher Ensembles (PATE) is demonstrated, in a black-box fashion, multiple models trained with disjoint datasets, such as records from different subsets of users, which achieves state-of-the-art privacy/utility trade-offs on MNIST and SVHN.
...
1
2
3
4
5
...