A simple sketching algorithm for entropy estimation over streaming data

@inproceedings{Clifford2013ASS,
  title={A simple sketching algorithm for entropy estimation over streaming data},
  author={Peter Clifford and Ioana Cosma},
  booktitle={AISTATS},
  year={2013}
}
We consider the problem of approximating the empirical Shannon entropy of a highfrequency data stream under the relaxed strict-turnstile model, when space limitations make exact computation infeasible. An equivalent measure of entropy is the Rényi entropy that depends on a constant α. This quantity can be estimated efficiently and unbiasedly from a low-dimensional synopsis called an α-stable data sketch via the method of compressed counting. An approximation to the Shannon entropy can be… CONTINUE READING