# Bayesian Inference on Principal Component Analysis Using Reversible Jump Markov Chain Monte Carlo

@inproceedings{Zhang2004BayesianIO, title={Bayesian Inference on Principal Component Analysis Using Reversible Jump Markov Chain Monte Carlo}, author={Zhihua Zhang and Kap Luk Chan and James Tin-Yau Kwok and Dit-Yan Yeung}, booktitle={AAAI}, year={2004} }

Based on the probabilistic reformulation of principal component analysis (PCA), we consider the problem of determining the number of principal components as a model selection problem. We present a hierarchical model for probabilistic PCA and construct a Bayesian inference method for this model using reversible jump Markov chain Monte Carlo (MCMC). By regarding each principal component as a point in a one-dimensional space and employing only birth-death moves in our reversible jump methodology…

## 17 Citations

### Choice of Dimension Using Reversible Jump Markov Chain Monte Carlo in the Multidimensional Scaling

- Computer Science2007 Chinese Control Conference
- 2006

A Reversible Jump Markov chain Monte Carlo (RJMCMC) algorithm is proposed for performing low-dimensional coordinate and choice of dimension simultaneously within the Bayesian framework.

### Bayesian nonparametric Principal Component Analysis

- Computer Science
- 2017

A Bayesian nonparametric principal component analysis (BNP-PCA) is introduced and is shown to be easily yet efficiently coupled with clustering or latent factor models within a unique framework.

### Automatic Model Selection by Cross-Validation for Probabilistic PCA

- Computer ScienceNeural Processing Letters
- 2009

This work presents a different way to solve this problem, where cross-validation and simulated annealing are combined to guide the search for an optimal model selection, providing a structured strategy to escape from suboptimal configurations.

### Bayesian principal component regression with data-driven component selection

- Mathematics
- 2012

Principal component regression (PCR) has two steps: estimating the principal components and performing the regression using these components. These steps generally are performed sequentially. In PCR,…

### Robust spatio-temporal latent variable models

- Computer Science
- 2011

This thesis introduces a variational Bayesian forward-backward algorithm based on Student-t assumptions that augment PPCA and ProbCCA respectively with autoregressive processes over the latent variables to additionally capture temporal relationships between the observations.

### A Bayesian Nonparametric Approach for the Analysis of Multiple Categorical Item Responses.

- Computer ScienceJournal of statistical planning and inference
- 2015

### Nonparametric Bayesian Sparse Factor Models with application to Gene Expression modelling

- Computer ScienceThe Annals of Applied Statistics
- 2011

A nonparametric Bayesian extension of Factor Analysis (FA) is proposed where observed data is modeled as a linear superposition of a potentially infinite number of hidden factors, including E. Coli and three biological data sets of increasing complexity.

### Small-variance asymptotics for non-parametric online robot learning

- Computer ScienceInt. J. Robotics Res.
- 2019

The generative model is used to synthesize both time-independent and time-dependent behaviors by relying on the principles of shared and autonomous control and yields a scalable online sequence clustering algorithm that is non-parametric in the number of clusters and the subspace dimension of each cluster.

### Perfect Gibbs Sampling of Order Constrained Non-IID Ordered Random Variates with Application to Bayesian Principal Components Analysis

- Mathematics, Computer Science
- 2021

A novel couplingfrom the past algorithm is proposed to “perfectly” (up to computable order of accuracy) simulate such order-constrained non-iid order statistics in Bayesian principal components analysis.

## References

SHOWING 1-10 OF 16 REFERENCES

### Reversible jump Markov chain Monte Carlo computation and Bayesian model determination

- Mathematics
- 1995

Markov chain Monte Carlo methods for Bayesian computation have until recently been restricted to problems where the joint distribution of all variables has a density with respect to some fixed…

### Bayesian analysis of mixture models with an unknown number of components- an alternative to reversible jump methods

- Mathematics
- 2000

Richardson and Green present a method of performing a Bayesian analysis of data from a finite mixture distribution with an unknown number of components. Their method is a Markov Chain Monte Carlo…

### Bayesian inference in hidden Markov models through the reversible jump Markov chain Monte Carlo method

- Mathematics, Computer Science
- 2000

It is shown how reversible jump Markov chain Monte Carlo techniques can be used to estimate the parameters as well as the number of components of a hidden Markov model in a Bayesian framework.

### On Bayesian Analysis of Mixtures with an Unknown Number of Components (with discussion)

- Mathematics
- 1997

New methodology for fully Bayesian mixture analysis is developed, making use of reversible jump Markov chain Monte Carlo methods that are capable of jumping between the parameter subspaces…

### Reversible jump, birth‐and‐death and more general continuous time Markov chain Monte Carlo samplers

- Mathematics
- 2003

Summary. Reversible jump methods are the most commonly used Markov chain Monte Carlo tool for exploring variable dimension statistical models. Recently, however, an alternative approach based on…

### Minimum-Entropy Data Partitioning Using Reversible Jump Markov Chain Monte Carlo

- Computer ScienceIEEE Trans. Pattern Anal. Mach. Intell.
- 2001

It is shown that an efficient partitioning may be given via a minimization of partition entropy and a reversible-jump sampling is introduced to explore the variable-dimension space of partition models.

### Bayesian PCA

- Computer ScienceNIPS
- 1998

This paper uses probabilistic reformulation as the basis for a Bayesian treatment of PCA to show that effective dimensionality of the latent space (equivalent to the number of retained principal components) can be determined automatically as part of the Bayesian inference procedure.

### Bayesian Radial Basis Functions of Variable Dimension

- Computer ScienceNeural Computation
- 1998

A Bayesian framework for the analysis of radial basis functions (RBF) is proposed that accommodates uncertainty in the dimension of the model, and posterior densities are computed using reversible jump Markov chain Monte Carlo samplers.

### Automatic Choice of Dimensionality for PCA

- Computer ScienceNIPS
- 2000

By interpreting PCA as density estimation, it is shown how to use Bayesian model selection to estimate the true dimensionality of the data, and the resulting estimate is simple to compute yet guaranteed to pick the correct dimensionality, given enough data.