Pure Differentially Private Summation from Anonymous Messages

@article{Ghazi2020PureDP,
  title={Pure Differentially Private Summation from Anonymous Messages},
  author={Badih Ghazi and Noah Golowich and Ravi Kumar and Pasin Manurangsi and R. Pagh and Ameya Velingker},
  journal={ArXiv},
  year={2020},
  volume={abs/2002.01919}
}
The shuffled (aka anonymous) model has recently generated significant interest as a candidate distributed privacy framework with trust assumptions better than the central model but with achievable errors smaller than the local model. We study pure differentially private (DP) protocols in the shuffled model for summation, a basic and widely used primitive: - For binary summation where each of n users holds a bit as an input, we give a pure $\epsilon$-DP protocol for estimating the number of… 

Figures from this paper

On the Power of Multiple Anonymous Messages
TLDR
A nearly tight lower bound on the error of locally-private frequency estimation in the low-privacy (aka high $\epsilon$) regime is obtained and implies that the protocols obtained from the amplification via shuffling work of Erlingsson et al. are essentially optimal for single-message protocols.
On Distributed Differential Privacy and Counting Distinct Elements
TLDR
The first lower bound for estimating the number of distinct elements provides the first $\omega(\sqrt(n)$ separation between global sensitivity and error in local differential privacy, thus answering an open question of Vadhan (2017).
Private Counting from Anonymous Messages: Near-Optimal Accuracy with Vanishing Communication Overhead
TLDR
Practical communication-efficient algorithms in the shuffled DP model for two basic aggregation primitives used in machine learning achieve accuracy that is arbitrarily close to that of central DP algorithms with an expected communication per user essentially matching what is needed without any privacy constraints.
Pure Differential Privacy from Secure Intermediaries
TLDR
This work gives a new protocol that ensures O(1/ε) error under pure differential privacy and shows how to use it to test uniformity of distributions over [d].
Private Summation in the Multi-Message Shuffle Model
TLDR
Two new protocols for summation in the shuffle model with improved accuracy and communication trade-offs are introduced, including a recursive construction based on the protocol from Balle et al. mentioned above and a novel analysis of the reduction from secure summation to shuffling introduced by Ishai etAl.
Connecting Robust Shuffle Privacy and Pan-Privacy
TLDR
It is found that robust approximate shuffle privacy and approximate pan-privacy have additive error $\Theta(\sqrt{k})$ for counting distinct elements and uniformity testing and that an $\Omega(k^{2/3})$ dependence is necessary for any robust pure shuffle private tester.
Distributed Differentially Private Averaging with Improved Utility and Robustness to Malicious Parties
TLDR
This work develops a novel distributed differentially private protocol which naturally scales with the number of parties and analyzes the differential privacy guarantees of the protocol and the impact of the graph topology, showing that it can match the accuracy of the trusted curator model even when each party communicates with only a logarithmic number of other parties chosen at random.
The limits of pan privacy and shuffle privacy for learning and estimation
TLDR
This work proves the first non-trivial lower bounds for high-dimensional learning and estimation in both the pan-private model and the general multi-message shuffle model.
DISTRIBUTED GAUSSIAN DIFFERENTIAL PRIVACY VIA SHUFFLING
  • Kan Chen
  • Computer Science, Mathematics
  • 2021
TLDR
It is proved that compared with the original shuffled model from Cheu et al. (2019), f -DP provides a tighter upper bound in terms of the privacy analysis of sum queries, which can be applied to broader classes of models to achieve more accurate privacy analysis.
The Fundamental Price of Secure Aggregation in Differentially Private Federated Learning
TLDR
This work theoretically and empirically specifies the fundamental price of using SecAgg and shows that the communication cost can be reduced significantly to under 1.2 bits per parameter in realistic privacy settings without decreasing test-time performance.
...
...

References

SHOWING 1-10 OF 45 REFERENCES
On the Power of Multiple Anonymous Messages
TLDR
A nearly tight lower bound on the error of locally-private frequency estimation in the low-privacy (aka high $\epsilon$) regime is obtained and implies that the protocols obtained from the amplification via shuffling work of Erlingsson et al. are essentially optimal for single-message protocols.
Differentially Private Summation with Multi-Message Shuffling
TLDR
This note shows a protocol with O(1/\epsilon)$ error and $O(\log(n/\delta)$ messages of size $O(n)$ per party, based on the work of Ishai et al.\ (FOCS 2006) showing how to implement distributed summation from secure shuffling.
Distributed Differential Privacy via Shuffling
TLDR
Evidence that the power of the shuffled model lies strictly between those of the central and local models is given: for a natural restriction of the model, it is shown that shuffled protocols for a widely studied selection problem require exponentially higher sample complexity than do central-model protocols.
Improved Summation from Shuffling
TLDR
Improved analysis achieving a dependency of the form $O(1+\sigma/\log n)$ addresses the intuitive question left open by Ishai et al. of whether the shuffling step in their protocol provides a "hiding in the crowd" amplification effect as $n$ increases.
Optimal Lower Bound for Differentially Private Multi-party Aggregation
TLDR
It is shown that any n-party protocol computing the sum with sparse communication graph must incur an additive error of $\Omega(\sqrt{n})$ with constant probability, in order to defend against potential coalitions of compromised users.
Local, Private, Efficient Protocols for Succinct Histograms
TLDR
Efficient protocols and matching accuracy lower bounds for frequency estimation in the local model for differential privacy are given and it is shown that each user need only send 1 bit to the server in a model with public coins.
Amplification by Shuffling: From Local to Central Differential Privacy via Anonymity
TLDR
It is shown, via a new and general privacy amplification technique, that any permutation-invariant algorithm satisfying e-local differential privacy will satisfy [MATH HERE]-central differential privacy.
Scalable and Differentially Private Distributed Aggregation in the Shuffled Model
TLDR
A simple and more efficient protocol for aggregation in the shuffled model, where communication as well as error increases only polylogarithmically in the number of users, is proposed.
Our Data, Ourselves: Privacy Via Distributed Noise Generation
TLDR
This work provides efficient distributed protocols for generating shares of random noise, secure against malicious participants, and introduces a technique for distributing shares of many unbiased coins with fewer executions of verifiable secret sharing than would be needed using previous approaches.
Heavy Hitters and the Structure of Local Privacy
We present a new locally differentially private algorithm for the heavy hitters problem which achieves optimal worst-case error as a function of all standardly considered parameters. Prior work
...
...