• Corpus ID: 2065910

Bayesian Inference on Principal Component Analysis Using Reversible Jump Markov Chain Monte Carlo

  title={Bayesian Inference on Principal Component Analysis Using Reversible Jump Markov Chain Monte Carlo},
  author={Zhihua Zhang and Kap Luk Chan and James Tin-Yau Kwok and Dit-Yan Yeung},
Based on the probabilistic reformulation of principal component analysis (PCA), we consider the problem of determining the number of principal components as a model selection problem. We present a hierarchical model for probabilistic PCA and construct a Bayesian inference method for this model using reversible jump Markov chain Monte Carlo (MCMC). By regarding each principal component as a point in a one-dimensional space and employing only birth-death moves in our reversible jump methodology… 

Figures and Tables from this paper

Choice of Dimension Using Reversible Jump Markov Chain Monte Carlo in the Multidimensional Scaling

A Reversible Jump Markov chain Monte Carlo (RJMCMC) algorithm is proposed for performing low-dimensional coordinate and choice of dimension simultaneously within the Bayesian framework.

Bayesian nonparametric Principal Component Analysis

A Bayesian nonparametric principal component analysis (BNP-PCA) is introduced and is shown to be easily yet efficiently coupled with clustering or latent factor models within a unique framework.

Automatic Model Selection by Cross-Validation for Probabilistic PCA

This work presents a different way to solve this problem, where cross-validation and simulated annealing are combined to guide the search for an optimal model selection, providing a structured strategy to escape from suboptimal configurations.

Bayesian principal component regression with data-driven component selection

Principal component regression (PCR) has two steps: estimating the principal components and performing the regression using these components. These steps generally are performed sequentially. In PCR,

Variational Bayesian functional PCA

  • A. Linde
  • Computer Science
    Comput. Stat. Data Anal.
  • 2008

Robust spatio-temporal latent variable models

This thesis introduces a variational Bayesian forward-backward algorithm based on Student-t assumptions that augment PPCA and ProbCCA respectively with autoregressive processes over the latent variables to additionally capture temporal relationships between the observations.

Nonparametric Bayesian Sparse Factor Models with application to Gene Expression modelling

A nonparametric Bayesian extension of Factor Analysis (FA) is proposed where observed data is modeled as a linear superposition of a potentially infinite number of hidden factors, including E. Coli and three biological data sets of increasing complexity.

Small-variance asymptotics for non-parametric online robot learning

The generative model is used to synthesize both time-independent and time-dependent behaviors by relying on the principles of shared and autonomous control and yields a scalable online sequence clustering algorithm that is non-parametric in the number of clusters and the subspace dimension of each cluster.

Perfect Gibbs Sampling of Order Constrained Non-IID Ordered Random Variates with Application to Bayesian Principal Components Analysis

A novel couplingfrom the past algorithm is proposed to “perfectly” (up to computable order of accuracy) simulate such order-constrained non-iid order statistics in Bayesian principal components analysis.



Reversible jump Markov chain Monte Carlo computation and Bayesian model determination

Markov chain Monte Carlo methods for Bayesian computation have until recently been restricted to problems where the joint distribution of all variables has a density with respect to some fixed

Bayesian analysis of mixture models with an unknown number of components- an alternative to reversible jump methods

Richardson and Green present a method of performing a Bayesian analysis of data from a finite mixture distribution with an unknown number of components. Their method is a Markov Chain Monte Carlo

Bayesian inference in hidden Markov models through the reversible jump Markov chain Monte Carlo method

It is shown how reversible jump Markov chain Monte Carlo techniques can be used to estimate the parameters as well as the number of components of a hidden Markov model in a Bayesian framework.

On Bayesian Analysis of Mixtures with an Unknown Number of Components (with discussion)

New methodology for fully Bayesian mixture analysis is developed, making use of reversible jump Markov chain Monte Carlo methods that are capable of jumping between the parameter subspaces

Reversible jump, birth‐and‐death and more general continuous time Markov chain Monte Carlo samplers

Summary. Reversible jump methods are the most commonly used Markov chain Monte Carlo tool for exploring variable dimension statistical models. Recently, however, an alternative approach based on

Minimum-Entropy Data Partitioning Using Reversible Jump Markov Chain Monte Carlo

It is shown that an efficient partitioning may be given via a minimization of partition entropy and a reversible-jump sampling is introduced to explore the variable-dimension space of partition models.

Bayesian PCA

This paper uses probabilistic reformulation as the basis for a Bayesian treatment of PCA to show that effective dimensionality of the latent space (equivalent to the number of retained principal components) can be determined automatically as part of the Bayesian inference procedure.

Bayesian Radial Basis Functions of Variable Dimension

A Bayesian framework for the analysis of radial basis functions (RBF) is proposed that accommodates uncertainty in the dimension of the model, and posterior densities are computed using reversible jump Markov chain Monte Carlo samplers.

Automatic Choice of Dimensionality for PCA

By interpreting PCA as density estimation, it is shown how to use Bayesian model selection to estimate the true dimensionality of the data, and the resulting estimate is simple to compute yet guaranteed to pick the correct dimensionality, given enough data.

Model selection by MCMC computation