# Physics of the Shannon limits

@inproceedings{Merhav2009PhysicsOT, title={Physics of the Shannon limits}, author={N. Merhav}, year={2009} }

We provide a simple physical interpretation, in the context of the second law of thermodynamics, to the information inequality (a.k.a. the Gibbs' inequality, which is also equivalent to the log-sum inequality), asserting that the relative entropy between two probability distributions cannot be negative. Since this inequality stands at the basis of the data processing theorem (DPT), and the DPT in turn is at the heart of most, if not all, proofs of converse theorems in Shannon theory, it is… Expand

#### 2 Citations

Asymptotic Analysis of MAP Estimation via the Replica Method and Applications to Compressed Sensing

- Mathematics, Computer Science
- IEEE Transactions on Information Theory
- 2012

#### References

SHOWING 1-9 OF 9 REFERENCES

Nonequilibrium fluctuation theorems in the presence of local heating.

- Mathematics, Physics
- Physical review. E, Statistical, nonlinear, and soft matter physics
- 2008

Illustrative example of the relationship between dissipation and relative entropy.

- Mathematics, Physics
- Physical review. E, Statistical, nonlinear, and soft matter physics
- 2009

Mutual information and minimum mean-square error in Gaussian channels

- Mathematics, Computer Science
- IEEE Transactions on Information Theory
- 2005