• Corpus ID: 202787279

On Tractable Computation of Expected Predictions

@inproceedings{Khosravi2019OnTC,
  title={On Tractable Computation of Expected Predictions},
  author={Pasha Khosravi and YooJung Choi and Yitao Liang and Antonio Vergari and Guy Van den Broeck},
  booktitle={NeurIPS},
  year={2019}
}
Computing expected predictions of discriminative models is a fundamental task in machine learning that appears in many interesting applications such as fairness, handling missing values, and data analysis. Unfortunately, computing expectations of a discriminative model with respect to a probability distribution defined by an arbitrary generative model has been proven to be hard in general. In fact, the task is intractable even for simple models such as logistic regression and a naive Bayes… 

Figures and Topics from this paper

Tractable computation of expected kernels
TLDR
This work constructs a circuit representation for kernels and proposes an approach to such tractable computation, and demonstrates possible advancements for kernel embedding frameworks by exploiting tractable expected kernels to derive new algorithms.
Tractable Computation of Expected Kernels by Circuits
TLDR
This work constructs a circuit representation for kernels and proposes an approach to tractable computation, and demonstrates possible advancements for kernel embedding frameworks by exploiting tractable expected kernels to derive new algorithms for two challenging scenarios.
Tractable Regularization of Probabilistic Circuits
TLDR
This work re-think regularization for PCs and proposes two intuitive techniques, data softening and entropy regularization, that both take advantage of PCs’ tractability and still have an efficient implementation as a computation graph.
Cautious Classification with Data Missing Not at Random Using Generative Random Forests
TLDR
This work develops an efficient algorithm for assessing the robustness of classifications made by probabilistic circuits to imputations of the nonignorable portion of missing data at prediction time and shows that the algorithm is exact when the model satisfies certain constraints.
A Compositional Atlas of Tractable Circuit Operations: From Simple Transformations to Complex Information-Theoretic Queries
TLDR
This paper characterize the tractability of a vocabulary of simple transformations in terms of sufficient structural constraints of the circuits they operate on, and derives a unified framework for reasoning about tractable models that generalizes several results in the literature and opens up novel tractable inference scenarios.
Nearest Neighbor Classifiers over Incomplete Information: From Certain Answers to Certain Predictions
TLDR
This paper proposes the notion of “Certain Predictions” (CP) — a test data example can be certainly predicted (CP’ed) if all possible classifiers trained on top of all possible worlds induced by the incompleteness of data would yield the same prediction.
Handling Missing Data in Decision Trees: A Probabilistic Approach
TLDR
This paper tackles the problem of handling missing data in decision trees by taking a probabilistic approach, and uses tractable density estimators to compute the "expected prediction" of the models of these models.
Nearest Neighbor Classifiers over Incomplete Information: From Certain Answers to Certain Predictions
TLDR
It is shown that the proposed CPClean approach built based on CP can often significantly outperform existing techniques in terms of classification accuracy with mild manual cleaning effort.
A Compositional Atlas of Tractable Circuit Operations for Probabilistic Inference
Circuit representations are becoming the lingua franca to express and reason about tractable generative and discriminative models. In this paper, we show how complex inference scenarios for these
Probabilistic Circuits for Variational Inference in Discrete Graphical Models
TLDR
This paper proposes a new approach that leverages the tractability of probabilistic circuit models, such as Sum Product Networks (SPN), to compute ELBO gradients exactly (without sampling) for a certain class of densities, and shows that selective-SPNs are suitable as an expressive variational distribution.
...
1
2
3
...

References

SHOWING 1-10 OF 42 REFERENCES
What to Expect of Classifiers? Reasoning about Logistic Regression with Missing Features
TLDR
This paper proposes a novel framework that classifies examples with missing features by computing the expected prediction with respect to a feature distribution, and uses geometric programming to learn a naive Bayes distribution that embeds a given logistic regression classifier and can efficiently take its expected predictions.
Tractable Learning for Structured Probability Spaces: A Case Study in Learning Preference Distributions
TLDR
This case study illustrates the effectiveness and flexibility of the developed PSDD framework as a domain-independent tool for learning and reasoning with structured probability spaces.
A Unified Approach to Interpreting Model Predictions
TLDR
A unified framework for interpreting predictions, SHAP (SHapley Additive exPlanations), which unifies six existing methods and presents new methods that show improved computational performance and/or better consistency with human intuition than previous approaches.
On the Hardness of Approximate Reasoning
  • D. Roth
  • Mathematics, Computer Science
    IJCAI
  • 1993
TLDR
It is proved that counting satisfying assignments of propositional languages is intractable even for Horn and monotone formulae, and even when the size of clauses and number of occurrences of the variables are extremely limited.
Cutset Networks: A Simple, Tractable, and Scalable Approach for Improving the Accuracy of Chow-Liu Trees
TLDR
Compared to approaches for learning other tractable models such as thin-junction trees, latent tree models, arithmetic circuits and sum-product networks, the proposed cutset networks approach is significantly more scalable, and provides similar or better accuracy.
Towards Compact Interpretable Models : Shrinking of Learned Probabilistic Sentential Decision Diagrams
Probabilistic sentential decision diagrams (PSDDs) were recently introduced as a tractable and interpretable representation of discrete probability distributions. PSDDs are tractable because they
Optimal Feature Selection for Decision Robustness in Bayesian Networks
TLDR
This work proposes the first algorithm to compute the expected same-decision probability for general Bayesian network classifiers, based on compiling the network into a tractable circuit representation, and develops a search algorithm for optimal feature selection that utilizes efficient incremental circuit modifications.
Towards Hardware-Aware Tractable Learning of Probabilistic Models
TLDR
A novel resource-aware cost metric is proposed that takes into consideration the hardware's properties in determining whether the inference task can be efficiently deployed, and a strategy that selects the device-settings that can optimally meet users' requirements is proposed.
"Why Should I Trust You?": Explaining the Predictions of Any Classifier
TLDR
LIME is proposed, a novel explanation technique that explains the predictions of any classifier in an interpretable and faithful manner, by learning aninterpretable model locally varound the prediction.
A Tractable Probabilistic Model for Subset Selection
TLDR
This paper proposes a tractable probabilistic model for subset selection and shows how it can be learned from data, is interpretable and subsumes a previously introduced model based on logistic regression.
...
1
2
3
4
5
...