SNAP: Efficient Extraction of Private Properties with Poisoning

  title={SNAP: Efficient Extraction of Private Properties with Poisoning},
  author={Harsh Chaudhari and Jackson Abascal and Alina Oprea and Matthew Jagielski and Florian Tram{\`e}r and Jonathan Ullman},
Property inference attacks allow an adversary to extract global properties of the training dataset from a machine learning model. Such attacks have privacy implications for data owners who share their datasets to train machine learning models. Several existing approaches for property inference attacks against deep neural networks have been proposed [1]–[3], but they all rely on the attacker training a large number of shadow models, which induces large computational overhead. In this paper, we… 

SafeNet: The Unreasonable Effectiveness of Ensembles in Private Collaborative Learning

It is argued that model ensembles, implemented in the framework called SafeNet, are a highly MPC-amenable way to avoid many adversarial ML attacks and the simplicity, cheap setup, and robustness properties of ensembling make it a strong choice for training ML models privately in MPC.



Property Inference from Poisoning

It is shown that poisoning attacks can boost the information leakage significantly and should be considered as a stronger threat model in sensitive applications where some of the data sources may be malicious.

Property Inference Attacks Against GANs

This paper proposes the first set of training dataset property inference attacks against GANs and proposes a general attack pipeline that can be tailored to two attack scenarios, including the full black-box setting and partial black- box setting and a novel optimization framework to increase the attack efficacy.

Subpopulation Data Poisoning Attacks

It is proved that, under some assumptions, subpopulation attacks are impossible to defend against, and empirically demonstrate the limitations of existing defenses against the authors' attacks, highlighting the difficulty of protecting machine learning against this threat.

Certified Defenses for Data Poisoning Attacks

This work addresses the worst-case loss of a defense in the face of a determined attacker by constructing approximate upper bounds on the loss across a broad family of attacks, for defenders that first perform outlier removal followed by empirical risk minimization.

Witches' Brew: Industrial Scale Data Poisoning via Gradient Matching

This work analyzes an attack that successfully works against new, randomly initialized models, and is nearly imperceptible to humans, all while perturbing only a small fraction of the training data, concluding that data poisoning is a credible threat, even for large-scale deep learning systems.

Exploiting Unintended Feature Leakage in Collaborative Learning

This work shows that an adversarial participant can infer the presence of exact data points -- for example, specific locations -- in others' training data and develops passive and active inference attacks to exploit this leakage.

Membership Inference Attacks Against Machine Learning Models

This work quantitatively investigates how machine learning models leak information about the individual data records on which they were trained and empirically evaluates the inference techniques on classification models trained by commercial "machine learning as a service" providers such as Google and Amazon.

Manipulating Machine Learning: Poisoning Attacks and Countermeasures for Regression Learning

A theoretically-grounded optimization framework specifically designed for linear regression and its effectiveness on a range of datasets and models is demonstrated and formal guarantees about its convergence and an upper bound on the effect of poisoning attacks when the defense is deployed are provided.

Reconstructing Training Data with Informed Adversaries

This work provides an effective reconstruction attack that model developers can use to assess memorization of individual points in general settings beyond those considered in previous works; it shows that standard models have the capacity to store enough information to enable high-fidelity reconstruction of training data points; and it demonstrates that differential privacy can successfully mitigate such attacks in a parameter regime where utility degradation is minimal.

Formalizing and Estimating Distribution Inference Risks

This work proposes a formal definition of distribution inference attacks general enough to describe a broad class of attacks distinguishing between possible training distributions, and introduces a metric that quantifies observed leakage by relating it to the leakage that would occur if samples from the training distribution were provided directly to the adversary.