• Corpus ID: 203951375

Estimating Density Models with Complex Truncation Boundaries

@article{Liu2019EstimatingDM,
  title={Estimating Density Models with Complex Truncation Boundaries},
  author={Song Liu and Takafumi Kanamori},
  journal={ArXiv},
  year={2019},
  volume={abs/1910.03834}
}
Truncated densities are probability density functions defined on truncated input domains. These densities share the same parametric form with their non-truncated counterparts up to a normalization term. However, normalization terms usually cannot be obtained in closed form for these distributions, due to complicated truncation domains. Score Matching is a powerful tool for fitting parameters in unnormalized models. However, it cannot be straightforwardly applied here as boundary conditions used… 
Generalized score matching for general domains
Estimation of density functions supported on general domains arises when the data is naturally restricted to a proper subset of the real space. This problem is complicated by typically intractable
Score Matched Conditional Exponential Families for Likelihood-Free Inference∗
To perform Bayesian inference for stochastic simulator models for which the likelihood is not accessible, Likelihood-Free Inference (LFI) relies on simulations from the model. Standard LFI methods
Interaction Models and Generalized Score Matching for Compositional Data
Applications such as the analysis of microbiome data have led to renewed interest in statistical methods for compositional data, i.e., multivariate data in the form of probability vectors that
Generalised Kernel Stein Discrepancy(GKSD): A Unifying Approach for Non-parametric Goodness-of-fit Testing
  • Wenkai Xu
  • Mathematics
  • 2021
Non-parametric goodness-of-fit testing procedures based on kernel Stein discrepancies (KSD) are promising approaches to validate general unnormalised distributions in various scenarios. Existing
Interpretable Stein Goodness-of-fit Tests on Riemannian Manifold
TLDR
This study develops goodness-of-fit testing and interpretable model criticism methods for general distributions on Riemannian manifolds, including those with an intractable normalization constant, based on extensions of kernel Stein discrepancy.

References

SHOWING 1-10 OF 16 REFERENCES
Some extensions of score matching
  • A. Hyvärinen
  • Mathematics, Computer Science
    Comput. Stat. Data Anal.
  • 2007
TLDR
It is shown how to estimate non-normalized models defined in the non-negative real domain, i.e. R"+^n", and it is shown that the score matching estimator can be obtained in closed form for some exponential families.
Estimation of Non-Normalized Statistical Models by Score Matching
  • A. Hyvärinen
  • Computer Science, Mathematics
    J. Mach. Learn. Res.
  • 2005
TLDR
While the estimation of the gradient of log-density function is, in principle, a very difficult non-parametric problem, it is proved a surprising result that gives a simple formula that simplifies to a sample average of a sum of some derivatives of the log- density given by the model.
Generalized Score Matching for Non-Negative Data
TLDR
This paper gives a generalized form of score matching for non-negative data that improves estimation efficiency and addresses an overlooked inexistence problem by generalizing the regularized score matching method of Lin et al. (2016) and improving its theoretical guarantees fornon-negative Gaussian graphical models.
Graphical Models for Non-Negative Data Using Generalized Score Matching
TLDR
This paper gives a generalized form of score matching for non-negative data that improves estimation efficiency and generalizes the regularized score matching method of Lin et al. (2016) fornon-negative Gaussian graphical models, with improved theoretical guarantees.
Kernel density estimation with doubly truncated data
In some applications with astronomical and survival data, doubly truncated data are sometimes encountered. In this work we introduce kernel-type density estimation for a random variable which is
Minimum Stein Discrepancy Estimators
TLDR
This work establishes the consistency, asymptotic normality, and robustness of DKSD and DSM estimators, then derive stochastic Riemannian gradient descent algorithms for their efficient optimisation.
A Kernel Test of Goodness of Fit
TLDR
A nonparametric statistical test for goodness-of-fit is proposed: given a set of samples, the test determines how likely it is that these were generated from a target density function, taking the form of a V-statistic in terms of the log gradients of the target density and the kernel.
Score matching estimators for directional distributions
One of the major problems for maximum likelihood estimation in the well-established directional models is that the normalising constants can be difficult to evaluate. A new general method of "score
Estimating a Dirichlet distribution
The Dirichlet distribution and its compound variant, the Dirichlet-multinomial, are two of the most basic models for proportional data, such as the mix of vocabulary words in a text document. Yet the
Interpretation and Generalization of Score Matching
  • Siwei Lyu
  • Computer Science, Mathematics
    UAI
  • 2009
TLDR
This paper provides a formal link between maximum likelihood and score matching and develops a generalization of score matching, which shows that score matching finds model parameters that are more robust with noisy training data.
...
1
2
...