Information geometry on hierarchy of probability distributions

@article{Amari2001InformationGO,
title={Information geometry on hierarchy of probability distributions},
author={Shun-ichi Amari},
journal={IEEE Trans. Information Theory},
year={2001},
volume={47},
pages={1701-1711}
}

An exponential family or mixture family of probability distributions has a natural hierarchical structure. This paper gives an “orthogonal” decomposition of such a system based on information geometry. A typical example is the decomposition of stochastic dependency among a number of random variables. In general, they have a complex structure of dependencies. Pairwise dependency is easily represented by correlation, but it is more difficult to measure effects of pure triplewise or higher order… CONTINUE READING