The Homological Nature of Entropy

@article{Baudot2015TheHN,
  title={The Homological Nature of Entropy},
  author={Pierre Baudot and Daniel Bennequin},
  journal={Entropy},
  year={2015},
  volume={17},
  pages={3253-3318}
}
We propose that entropy is a universal co-homological class in a theory associated to a family of observable quantities and a family of probability distributions. Three cases are presented: (1) classical probabilities and random variables; (2) quantum probabilities and observable operators; (3) dynamic probabilities and observation trees. This gives rise to a new kind of topology for information processes, that accounts for the main information functions: entropy, mutual-informations at all… 
Information structures and their cohomology
We introduce the category of information structures, whose objects are suitable diagrams of measurable sets that encode the possible outputs of a given family of observables and their mutual
The structure of information: from probability to homology
TLDR
This text serves as a detailed introduction to information cohomology, containing the necessary background in probability theory and homological algebra, and proves that information cohmology is invariant under isomorphisms of generalized structures.
A New Perspective of Entropy
  • T. Bradley
  • Computer Science, Mathematics
    The Journal of The Math3ma Institute
  • 2022
TLDR
It is shown that entropy, abstract algebra, and topology are inextricably linked through a version of a well-known formula from calculus known as the Leibniz rule.
The Group Theoretic Roots of Information I: permutations, symmetry, and entropy
  • D. Galas
  • Mathematics, Computer Science
    ArXiv
  • 2019
TLDR
A combinatorial measure of information and disorder is proposed, in terms of integers and discrete functions, that it is shown that the integer entropy converges uniformly to the Shannon entropy when the group includes all permutations, the Symmetric group, and the number of objects increases without bound.
A homological characterization of generalized multinomial coefficients related to the entropic chain rule
TLDR
The asymptotic relationship between the multiplicative relations among multinomial coefficients and the (additive) recurrence property of Shannon entropy known as the chain rule is extended to a correspondence between certain generalized multin coefficients and any $\alpha$-entropy, that sheds new light on the meaning of the chainRule and its deformations.
Entropy as a Topological Operad Derivation
TLDR
The main result is that Shannon entropy defines a derivation of the operad of topological simplices, and that for every derivations of this operad there exists a point at which it is given by a constant multiple of Shannon entropy.
A functorial characterization of von Neumann entropy
TLDR
The von Neumann entropy is classified as a certain concave functor from finite-dimensional non-commutative probability spaces and state-preserving $*$-homomorphisms to real numbers and the existence of disintegrations for classical probability spaces plays a crucial role in this classification.
The Poincaré-Shannon Machine: Statistical Physics and Machine Learning Aspects of Information Cohomology
TLDR
The computationally tractable subcase of simplicial information cohomology represented by entropy Hk and information Ik landscapes and their respective paths is developed, allowing investigation of Shannon’s information in the multivariate case without the assumptions of independence or of identically distributed variables.
Unifying formalism for multivariate information-related measures: Möbius operators on subset lattices
Information-related measures are important tools of multi-variable data analysis, as measures of dependence among variables and the description of order in biological and physical systems.
Foliations-Webs-Hessian Geometry-Information Geometry-Entropy and Cohomology
TLDR
The theory of homology of Koszul-Vinberg algebroids and their modules (KV homology in short) is used for investigating links between differential information geometry and differential topology and the links between the classical theory of models, the new theory and Vanishing Theorems in the theory of Homological statistical models.
...
...

References

SHOWING 1-10 OF 68 REFERENCES
Topological forms of information
We propose that entropy is a universal co-homological class in a theory associated to a family of observable quantities and a family of probability distributions. Three cases are presented: 1)
Linear Dependence Structure of the Entropy Space
  • T. Han
  • Computer Science
    Inf. Control.
  • 1975
A Characterization of Entropy in Terms of Information Loss
TLDR
It is shown that Shannon entropy gives the only concept of information loss that is functorial, convex-linear and continuous and naturally generalizes to Tsallis entropy as well.
In a Search for a Structure, Part 1: On Entropy
Mathematics is about ”interesting structures”. What make a structure interesting is an abundance of interesting problems; we study a structure by solving these problems. The worlds of science, as
TRIPLES, ALGEBRAS AND COHOMOLOGY
It is with great pleasure that the editors of Theory and Applications of Categories make this dissertation generally available. Although the date on the thesis is 1967, there was a nearly complete
Elements of Information Theory
TLDR
The author examines the role of entropy, inequality, and randomness in the design of codes and the construction of codes in the rapidly changing environment.
Towards a Group-Theoretical Interpretation of Mechanics
We argue that the classical description of a symplectic manifold endowed with a Hamiltonian action of an abelian Lie group G and the corresponding quantum theory can be understood as different
Koszul duality for Operads
(0.1) The purpose of this paper is to relate two seemingly disparate developments. One is the theory of graph cohomology of Kontsevich [Kon 2 3] which arose out of earlier works of Penner [Pe] and
Quantum Probability for Probabilists
In recent years, the classical theory of stochastic integration and stochastic differential equations has been extended to a non-commutative set-up to develop models for quantum noises. The author, a
Emergences in Quantum Measurement Processes
Abstract Quantum mechanics is acknowledged as the fundamental theory on which the whole fabric of physics is supposed to rely. And yet, the features of quantum measurements, processes that provide
...
...