Optimal Differentially Private Learning of Thresholds and Quasi-Concave Optimization

@article{Cohen2022OptimalDP,
  title={Optimal Differentially Private Learning of Thresholds and Quasi-Concave Optimization},
  author={Edith Cohen and Xin Lyu and Jelani Nelson and Tam'as Sarl'os and Uri Stemmer},
  journal={Proceedings of the 55th Annual ACM Symposium on Theory of Computing},
  year={2022}
}
  • E. CohenXin Lyu Uri Stemmer
  • Published 11 November 2022
  • Computer Science
  • Proceedings of the 55th Annual ACM Symposium on Theory of Computing
The problem of learning threshold functions is a fundamental one in machine learning. Classical learning theory implies sample complexity of O(ξ−1 log(1/β)) (for generalization error ξ with confidence 1−β). The private version of the problem, however, is more challenging and in particular, the sample complexity must depend on the size |X| of the domain. Progress on quantifying this dependence, via lower and upper bounds, was made in a line of works over the past decade. In this paper, we… 
View on ACM
arxiv.org
4 Citations
Background Citations
1

4 Citations

Private Everlasting Prediction

This work introduces private everlasting prediction taking into account the privacy of both the training set and the (adaptively chosen) queries made to the predictor, and presents a generic construction of private everlasting predictors in the PAC model.

Differentially Private Medians and Interior Points for Non-Pathological Data

We construct differentially private estimators with low sample complexity that estimate the median of an arbitrary distribution over $\mathbb{R}$ satisfying very mild moment conditions. Our result…

The Target-Charging Technique for Privacy Accounting across Interactive Computations

TCT generalizes tools such as the sparse vector technique and top-$k$ selection from private candidates and extends their remarkable privacy enhancement benefits from noisy Lipschitz functions to general private algorithms.

Relaxed Models for Adversarial Streaming: The Advice Model and the Bounded Interruptions Model

This work set out to explore intermediate models that allow us to interpolate between the oblivious and the adversarial models, and put forward the following two models.

30 References

Privately Learning Thresholds: Closing the Exponential Gap

An improved version of the algorithm constructed for the related interior point problem, based on selecting an input-dependent hash function and using it to embed the database into a domain whose size is reduced logarithmically; this results in a new database which can be used to generate an interior point in the original database in a differentially private manner.

Differentially Private Release and Learning of Threshold Functions

The first nontrivial lower bound for releasing thresholds with (ε, δ) differential privacy is given, showing that the task is impossible over an infinite domain X, and moreover requires sample complexity n ≥ Ω(log* |X|), which grows with the size of the domain.

Bounds on the sample complexity for private learning and private data release

This work examines several private learning tasks and gives tight bounds on their sample complexity, and shows strong separations between sample complexities of proper and improper private learners (such separation does not exist for non-private learners), and between sample complexity of efficient and inefficient proper private learners.

Sample Complexity Bounds for Differentially Private Learning

An upper bound on the sample requirement of learning with label privacy is provided that depends on a measure of closeness between and the unlabeled data distribution and applies to the non-realizable as well as the realizable case.

On the Sample Complexity of Privately Learning Axis-Aligned Rectangles

This work revisits the fundamental problem of learning Axis-Aligned-Rectangles over a grid X d ⊆ R d with differential privacy, and presents a novel algorithm that reduces the sample complexity to only O d · (log ∗ | X | ) 1 .

Sample Complexity Bounds on Differentially Private Learning via Communication Complexity

It is shown that the sample complexity of learning with (pure) differential privacy can be arbitrarily higher than the samplecomplexity of learning without the privacy constraint or the sample complex oflearning with approximate differential privacy.

Private PAC learning implies finite Littlestone dimension

We show that every approximately differentially private learning algorithm (possibly improper) for a class H with Littlestone dimension d requires Ω(log*(d)) examples. As a corollary it follows that…

Learning Privately with Labeled and Unlabeled Examples

An alternative approach is suggested, inspired by the (non-private) models of semi-supervised learning and active-learning, where the focus is on the sample complexity of labeled examples whereas unlabeled examples are of a significantly lower cost.

Private Center Points and Learning of Halfspaces

The construction establishes a relationship between these two problems that is reminiscent of the relation between the median and learning one-dimensional thresholds, which suggests that the problem of privately locating a center point may have further applications in the design of differentially private algorithms.

Private Learning of Halfspaces: Simplifying the Construction and Reducing the Sample Complexity

We present a differentially private learner for halfspaces over a finite grid $G$ in $\mathbb{R}^d$ with sample complexity $\approx d^{2.5}\cdot 2^{\log^*|G|}$, which improves the state-of-the-art…