# Learning Distributions by Their Density Levels: A Paradigm for Learning without a Teacher

@article{BenDavid1997LearningDB, title={Learning Distributions by Their Density Levels: A Paradigm for Learning without a Teacher}, author={Shai Ben-David and Michael Lindenbaum}, journal={J. Comput. Syst. Sci.}, year={1997}, volume={55}, pages={171-182} }

We propose a mathematical model for learning the high-density areas of an unknown distribution from (unlabeled) random points drawn according to this distribution. While this type of a learning task has not been previously addressed in the computational learnability literature, we believe that this it a rather basic problem that appears in many practical learning scenarios. From a statistical theory standpoint, our model may be viewed as a restricted instance of the fundamental issue of…

## 67 Citations

### The Computational Complexity of Densest Region Detection

- Computer ScienceJ. Comput. Syst. Sci.
- 2000

A formal learning model for this task that uses a hypothesis class as it “anti-overfitting” mechanism is introduced and it is shown that for some constants, depending on the hypothesis class, these problems are NP-hard to approximate to within these constant factors.

### Pac Learning from Positive Statistical Queries ?

- Computer Science
- 1998

It is shown that k-DNF and k-decision lists are learnable in both models, i.e. with far less information than it is assumed in previously used algorithms.

### PAC Learning from Positive Statistical Queries

- Computer ScienceALT
- 1998

It is shown that k-DNF and k-decision lists are learnable in both models, i.e. with far less information than it is assumed in previously used algorithms.

### Learning Low Density Separators

- Computer ScienceAISTATS
- 2009

This work proposes two natural learning paradigms and proves that, on input random samples generated i.i.d. by any distribution, they are guaranteed to converge to the optimal separator for that distribution.

### PAC learning of probability distributions over a discrete domain

- Computer Science, MathematicsTheor. Comput. Sci.
- 2003

### Estimating the Support of a High-Dimensional Distribution

- Computer ScienceNeural Computation
- 2001

The algorithm is a natural extension of the support vector algorithm to the case of unlabeled data by carrying out sequential optimization over pairs of input patterns and providing a theoretical analysis of the statistical performance of the algorithm.

### Support Vector Method for Novelty Detection

- Computer ScienceNIPS
- 1999

The algorithm is a natural extension of the support vector algorithm to the case of unlabelled data and is regularized by controlling the length of the weight vector in an associated feature space.

### Accepted for NIPS ’ 99 SV Estimation of a Distribution ’ s Support

- Computer Science
- 1999

This work proposes an algorithm which approaches the problem of estimating a subset S of input space such that the probability that a test point drawn from P lies outside of S is bounded by some a priori specified 0 < 1.

### A Lower Bound for Learning Distributions Generated by Probabilistic Automata

- Computer ScienceALT
- 2010

A lower bound is shown: every algorithm to learn PDFA using queries with a resonable tolerance needs a number of queries larger than (1/µ)c for every c < 1.

## References

SHOWING 1-10 OF 25 REFERENCES

### On the learnability of discrete distributions

- Computer ScienceSTOC '94
- 1994

A new model of learning probability distributions from independent draws is introduced, inspired by the popular Probably Approximately Correct (PAC) model for learning boolean functions from labeled examples, in the sense that it emphasizes efficient and approximate learning, and it studies the learnability of restricted classes of target distributions.

### Probably Approximate Learning of Sets and Functions

- Computer Science, MathematicsSIAM J. Comput.
- 1991

The scope of the learning model is widened to include the inference of functions, and the Vapnik–Chervonenkis dimension is extended to obtain a measure called the “generalized dimension” of a class of functions.

### General bounds on the number of examples needed for learning probabilistic concepts

- Computer ScienceCOLT '93
- 1993

A new method for designing learning algorithms: dynamic partitioning of the domain by use of splitting trees is introduced and it can be shown that the resulting lower bounds for learning ND are tight to within a logarithmic factor.

### Efficient distribution-free learning of probabilistic concepts

- Computer ScienceProceedings [1990] 31st Annual Symposium on Foundations of Computer Science
- 1990

A model of machine learning in which the concept to be learned may exhibit uncertain or probabilistic behavior is investigated, and an underlying theory of learning p-concepts is developed in detail.

### Localization vs. Identification of Semi-Algebraic Sets

- Computer ScienceCOLT '93
- 1993

The authors' analysis provides a mathematical ground to the intuition that localization is indeed much easier than identification and upper-bounds on the hardness of localization are established by applying a new, algebraic-geometry based, general tool for the calculation of the VC-dimension of classes of algebraically defined objects.

### A general lower bound on the number of examples needed for learning

- Computer ScienceCOLT '88
- 1988

### Bounding the Vapnik-Chervonenkis Dimension of Concept Classes Parameterized by Real Numbers

- Computer Science, MathematicsCOLT '93
- 1993

The results show that for two general kinds of concept class the V-C dimension is polynomially bounded in the number of real numbers used to define a problem instance, and that in the continuous case, as in the discrete, the real barrier to efficient learning in the Occam sense is complexity- theoretic and not information-theoretic.

### Learnability and the Vapnik-Chervonenkis dimension

- Computer ScienceJACM
- 1989

This paper shows that the essential condition for distribution-free learnability is finiteness of the Vapnik-Chervonenkis dimension, a simple combinatorial parameter of the class of concepts to be learned.

### Lower Bounds for Sampling Algorithms for Estimating the Average

- Computer Science, MathematicsInf. Process. Lett.
- 1995