Estimating entropy on m bins given fewer than m samples

Consider a sequence p/sub N/ of discrete probability measures, supported on m/sub N/ points, and assume that we observe N independent and identically distributed (i.i.d.) samples from each p/sub N/. We demonstrate the existence of an estimator of the entropy, H(p/sub N/), which is consistent even if the ratio N/m/sub N/ is bounded (and, as a corollary, even… CONTINUE READING



Citations per Year

121 Citations

Semantic Scholar estimates that this publication has 121 citations based on the available data.

See our FAQ for additional information.