On the Latent Variable Interpretation in Sum-Product Networks

@article{Peharz2016OnTL,
  title={On the Latent Variable Interpretation in Sum-Product Networks},
  author={Robert Peharz and Robert Gens and Franz Pernkopf and Pedro M. Domingos},
  journal={IEEE Transactions on Pattern Analysis and Machine Intelligence},
  year={2016},
  volume={39},
  pages={2030-2044}
}
One of the central themes in Sum-Product networks (SPNs) is the interpretation of sum nodes as marginalized latent variables (LVs. [] Key Method We discuss conditional independencies in augmented SPNs, formally establish the probabilistic interpretation of the sum-weights and give an interpretation of augmented SPNs as Bayesian networks. Based on these results, we find a sound derivation of the EM algorithm for SPNs. Furthermore, the Viterbi-style algorithm for MPE proposed in literature was never proven to…
View on IEEE
ieeexplore.ieee.org
101 Citations
Highly Influential Citations
19
Background Citations
61
Methods Citations
39
Results Citations
5

101 Citations

Bayesian Learning of Sum-Product Networks

A well-principled Bayesian framework for SPN structure learning, which consistently and robustly learns SPN structures under missing data, and a natural parametrisation for an important and widely used special case of SPNs.

Sum–product graphical models

The theoretical and practical results demonstrate that jointly exploiting properties of SPNs and GMs is an interesting direction of future research and provide two applications of SPGMs in density estimation with empirical results close to or surpassing state-of-the-art models.

Parameter and Structure Learning Techniques for Sum Product Networks

A new Bayesian moment matching (BMM) algorithm to learn the parameters for SPNs generatively and a discriminative learning algorithm based on the Extended BaumWelch (EBW) algorithm is presented.

Structure Inference in Sum-Product Networks using Infinite Sum-Product Trees

This approach is the first correct and successful extension of SPNs to a Bayesian nonparametric model and shows that infinite SPTs can be used successfully to discover SPN structures and outperform infinite Gaussian mixture models in the task of density estimation.

Sum-Product Network Decompilation

SPN2BN is proposed, an algorithm that decompiles an SPN into a BN and can be precisely characterized with respect to the compiled BN, and it is established that the compilation-decompilation process is idempotent.

Collapsed Variational Inference for Sum-Product Networks

This work proposes a novel deterministic collapsed variational inference algorithm for SPNs that is computationally efficient, easy to implement and at the same time allows us to incorporate prior information into the optimization formulation.

Maximum A Posteriori Inference in Sum-Product Networks

This work reduces general MAP inference to its special case without evidence and hidden variables and shows that it is NP-hard to approximate the MAP problem to 2nε for fixed 0 ≤ ε < 1, where n is the input size.

Sum-Product Networks: A Survey

A survey of SPNs, including their definition, the main algorithms for inference and learning from data, several applications, a brief review of software libraries, and a comparison with related models are offered.

Robustifying sum-product networks

Explaining Deep Tractable Probabilistic Models: The sum-product network case

The notion of a context-specific independence tree (CSI-tree) is considered and an iterative algorithm that converts an SPN to a CSI-tree is presented, which is both interpretable and explainable to the domain expert.
...

45 References

On the Relationship between Sum-Product Networks and Bayesian Networks

This paper proves that every SPN can be converted into a BN in linear time and space in terms of the network size and introduces the notion of {\em normal} SPN and presents a theoretical analysis of the consistency and decomposability properties.

Structure Inference in Sum-Product Networks using Infinite Sum-Product Trees

This approach is the first correct and successful extension of SPNs to a Bayesian nonparametric model and shows that infinite SPTs can be used successfully to discover SPN structures and outperform infinite Gaussian mixture models in the task of density estimation.

Learning Sum-Product Networks with Direct and Indirect Variable Interactions

ID-SPN is presented, a new algorithm for learning SPN structure that unifies the combination of direct and indirect interactions that leads to significantly better accuracy than several state-of-the-art algorithms forlearning SPNs and other tractable models.

Learning the Structure of Sum-Product Networks

This work proposes the first algorithm for learning the structure of SPNs that takes full advantage of their expressiveness, and shows that the learned SPNs are typically comparable to graphical models in likelihood but superior in inference speed and accuracy.

A Unified Approach for Learning the Parameters of Sum-Product Networks

It is proved that any complete and decomposable SPN is equivalent to a mixture of trees where each tree corresponds to a product of univariate distributions, and the objective function when learning SPNs based on the maximum likelihood estimation (MLE) principle is characterized.

Simplifying, Regularizing and Strengthening Sum-Product Network Structure Learning

This work enhances one of the best structure learner, LearnSPN, aiming to improve both the structural quality of the learned networks and their achieved likelihoods, and proves its claims by empirically evaluating the learned SPNs on several benchmark datasets against other competitive SPN and PGM structure learners.

Learning Selective Sum-Product Networks

We consider the selectivity constraint on the structure of sum-product networks (SPNs), which allows each sum node to have at most one child with non-zero output for each possible input. This allows…

Context-Specific Independence in Bayesian Networks

This paper proposes a formal notion of context-specific independence (CSI), based on regularities in the conditional probability tables (CPTs) at a node, and proposes a technique, analogous to (and based on) d-separation, for determining when such independence holds in a given network.

On Theoretical Properties of Sum-Product Networks

It is shown that the weights of any complete and consistent SPN can be transformed into locally normalized weights without changing the SPN distribution, and that consistent SPNs cannot model distributions significantly (exponentially) more compactly than decomposable SPNs.

Language modeling with sum-product networks

It is shown that the ability of SPN to use hidden layers to model complex dependencies among words, and its tractable inference and learning times, make it a suitable framework for a language model.