On the hardness of entropy minimization and related problems

  title={On the hardness of entropy minimization and related problems},
  author={Mladen Kovacevic and Ivan Stanojevic and Vojin Senk},
  journal={2012 IEEE Information Theory Workshop},
We investigate certain optimization problems for Shannon information measures, namely, minimization of joint and conditional entropies H(X, Y), H(X|Y), H(Y|X), and maximization of mutual information I(X; Y), over convex regions. When restricted to the so-called transportation polytopes (sets of distributions with fixed marginals), very simple proofs of NP-hardness are obtained for these problems because in that case they are all equivalent, and their connection to the well-known SUBSET SUM and… CONTINUE READING


Publications citing this paper.
Showing 1-7 of 7 extracted citations


Publications referenced by this paper.
Showing 1-10 of 12 references

Discrete Optimization: An Algorithmic Theory

S. Onn, Nonlinear
European Mathematical Society, • 2010
View 2 Excerpts

and W

X. He, L. Deng
Chou, ”Discriminative Learning in S equential Pattern Recognition”,IEEE Signal Processing Magazine, vol. 25, pp. 14–36 • 2008

Combinatorial Matrix Classes

R. A. Brualdi
Cambridge University Press • 2006
View 1 Excerpt

and H

J. N. Kapur, G. Baciu
K. Kesavan, ”The MinMax Info rmation Measure”,Int. J. Systems Sci., vol. 26, pp. 1–12 • 1995
View 2 Excerpts

Complexity, Addison-Wesley Publishing

C H. Papadimitriou, Computational
View 2 Excerpts

Maximum-Entropy Models in Science and Engineering

J. N. Kapur
New Delhi, India: Wiley • 1989

Similar Papers

Loading similar papers…