• Corpus ID: 669957

A Bayesian Characterization of Relative Entropy

  title={A Bayesian Characterization of Relative Entropy},
  author={John C. Baez and Tobias Fritz},
  • J. Baez, T. Fritz
  • Published 13 February 2014
  • Mathematics, Computer Science, Physics
  • ArXiv
We give a new characterization of relative entropy, also known as the Kullback-Leibler divergence. We use a number of interesting categories related to probability theory. In particular, we consider a category FinStat where an object is a finite set equipped with a probability distribution, while a morphism is a measure-preserving function $f: X \to Y$ together with a stochastic right inverse $s: Y \to X$. The function $f$ can be thought of as a measurement process, while s provides a… 
A functorial characterization of von Neumann entropy
The von Neumann entropy is classified as a certain concave functor from finite-dimensional non-commutative probability spaces and state-preserving $*$-homomorphisms to real numbers and the existence of disintegrations for classical probability spaces plays a crucial role in this classification.
A Probability Monad as the Colimit of Finite Powers
It is proved that this Kantorovich monad arises from a colimit construction on finite powers, which formalizes the intuition that probability measures are limits of finite samples and allows for the development of integration theory and other things.
A Probability Monad as the Colimit of Spaces of Finite Samples
We define and study a probability monad on the category of complete metric spaces and short maps. It assigns to each space the space of Radon probability measures on it with finite first moment,
Towards a functorial description of quantum relative entropy
Preliminary calculations suggest that the finite-dimensional quantum (Umegaki) relative entropy might be characterized in a similar way to the classical relative entropy, and it is explicitly proved that it defines an affine functor in the special case where the relative entropy is finite.
A non-commutative Bayes' theorem.
Using a diagrammatic reformulation of Bayes' theorem, we provide a necessary and sufficient condition for the existence of Bayesian inference in the setting of finite-dimensional $C^*$-algebras. In
Dirichlet Polynomials and Entropy
The main result of this paper is the following: the rectangle-area formula A(d) = L (d)W( d) holds for any Dirichlet polynomial d, and the entropy of an empirical distribution can be calculated entirely in terms of the homomorphism h applied to its corresponding Dirichlets.
Stinespring's construction as an adjunction
Given a representation of a unital $C^*$-algebra $\mathcal{A}$ on a Hilbert space $\mathcal{H}$, together with a bounded linear map $V:\mathcal{K}\to\mathcal{H}$ from some other Hilbert space, one
The Design of Global Correlation Quantifiers and Continuous Notions of Statistical Sufficiency
Using first principles from inference, a set of functionals are found to be uniquely capable of determining whether a certain class of inferential transformations, ρ→∗ρ′, preserve, destroy or create correlations.
Non-commutative disintegrations: existence and uniqueness in finite dimensions
We utilize category theory to define non-commutative disintegrations, regular conditional probabilities, and optimal hypotheses for finite-dimensional C*-algebras. In the process, we introduce a
Postquantum Br\`{e}gman relative entropies and nonlinear resource theories
We introduce the family of postquantum Br\`{e}gman relative entropies, based on nonlinear embeddings into reflexive Banach spaces (with examples given by reflexive noncommutative Orlicz spaces over


A Characterization of Entropy in Terms of Information Loss
It is shown that Shannon entropy gives the only concept of information loss that is functorial, convex-linear and continuous and naturally generalizes to Tsallis entropy as well.
Convex Spaces I: Definition and Examples
We propose an abstract definition of convex spaces as sets where one can take convex combinations in a consistent way. A priori, a convex space is an algebra over a finitary version of the Giry
Quantum Entropy and Its Use
I Entropies for Finite Quantum Systems.- 1 Fundamental Concepts.- 2 Postulates for Entropy and Relative Entropy.- 3 Convex Trace Functions.- II Entropies for General Quantum Systems.- 4 Modular
An Introduction to the Theory of Functional Equations and Inequalities
Preliminaries.- Set Theory.- Topology.- Measure Theory.- Algebra.- Cauchy's Functional Equation and Jensen's Inequality.- Additive Functions and Convex Functions.- Elementary Properties of Convex
Bayesian surprise attracts human attention
A formal Bayesian definition of surprise is proposed to capture subjective aspects of sensory information and it is shown that Bayesian surprise is a strong attractor of human attention, with 72% of all gaze shifts directed towards locations more surprising than the average.
Higher Operads, Higher Categories
Part I. Background: 1. Classical categorical structures 2. Classical operads and multicategories 3. Notions of monoidal category Part II. Operads. 4. Generalized operads and multicategories: basics
An operadic introduction to entropy, The n-Category Café, 18 May 2011. Available at http://golem.ph.utexas.edu/category/2011/05/an operadic introduction to en.html
  • 2011
Higher Operads
  • Higher Categories, London Mathematical Society Lecture Note Series 298, Cambridge U. Press, Cambridge
  • 2004
An operadic introduction to entropy, The n-Category Café
  • An operadic introduction to entropy, The n-Category Café
  • 2011
Baldi , Bayesian surprise attracts human attention
  • Advances in Neural Information Processing Systems
  • 2005