# Model Criticism in Latent Space

@article{Seth2017ModelCI,
title={Model Criticism in Latent Space},
author={Sohan Seth and Iain Murray and Christopher K. I. Williams},
journal={Bayesian Analysis},
year={2017}
}
• Published 13 November 2017
• Computer Science
• Bayesian Analysis
Model criticism is usually carried out by assessing if replicated data generated under the fitted model looks similar to the observed data, see e.g. Gelman, Carlin, Stern, and Rubin [2004, p. 165]. This paper presents a method for latent variable models by pulling back the data into the space of latent variables, and carrying out model criticism in that space. Making use of a model's structure enables a more direct assessment of the assumptions made in the prior and likelihood. We demonstrate…

## Figures and Tables from this paper

• Computer Science
• 2017
A method to evaluate the specified model class by assessing its capability of reproducing data that is similar to the observed data record is developed, based on the information-theoretic properties of models viewed as data generators and is applicable to e.g. sequential data and nonlinear dynamical models.
• Computer Science
EMNLP
• 2022
This work proposes to apply a statistical tool, model criticism in latent space, to evaluate the high-level structure of the generated text and finds that transformer-based language models are able to capture topical structures but have a harder time maintaining structural coherence or modeling coreference.
• Mathematics
• 2019
This work considers checking for prior-data conflict in Bayesian models by expanding the prior used for the analysis into a larger family of priors, and considering a marginal likelihood score statistic for the expansion parameter.
• Mathematics
ICML
• 2021
This study develops goodness-of-fit testing and interpretable model criticism methods for general distributions on Riemannian manifolds, including those with an intractable normalization constant, based on extensions of kernel Stein discrepancy.
• Computer Science
ArXiv
• 2019
It is shown that hierarchical multi-task dynamical systems (MTDSs) provide direct user control over sequence generation, via use of a latent code $\mathbf{z}$ that specifies the customization to the individual data sequence that enables style transfer, interpolation and morphing within generated sequences.

## References

SHOWING 1-10 OF 33 REFERENCES

• Computer Science
• 1998
A mixture prior density with two beta distributed components is used to expand the model in a meaningful way and it is concluded that a relatively at prior distribution is inappropriate.
• Computer Science
NIPS
• 2015
An exploratory approach to statistical model criticism using maximum mean discrepancy (MMD) two sample tests is proposed and it is demonstrated on synthetic data that the selected statistic can be used to identify where a statistical model most misrepresents the data it was trained on.
Predictive checking functions for transformation, serial correlation, bad values, and their relation with Bayesian options are considered, and robustness is seen from a Bayesian viewpoint and examples are given.
• Mathematics, Computer Science
• 2000
This paper proposes two alternatives for computing a p value, the conditional predictive p value and the partial posterior predictive pvalue, and indicates their advantages from both Bayesian and frequentist perspectives.
• Psychology
Statistics in medicine
• 1995
Four mixture models are fit within a Bayesian model monitoring using posterior predictive checks framework, where the distinctions between models arise from assumptions about the variance of the shifted observations and the exchangeability of schizophrenic individuals.
• Mathematics
• 1996
This paper considers Bayesian counterparts of the classical tests for good- ness of fit and their use in judging the fit of a single Bayesian model to the observed data. We focus on posterior
• J. Kruschke
• Political Science
Wiley interdisciplinary reviews. Cognitive science
• 2010
A fatal flaw of NHST is reviewed and some benefits of Bayesian data analysis are introduced and illustrative examples of multiple comparisons in Bayesian analysis of variance and Bayesian approaches to statistical power are presented.
• Biology
Proceedings of the National Academy of Sciences
• 2009
This work provides a statistical interpretation to current developments in likelihood-free Bayesian inference that explicitly accounts for discrepancies between the model and the data, termed Approximate Bayesian Computation under model uncertainty (ABCμ).