Probabilistic Principal Component Analysis

  title={Probabilistic Principal Component Analysis},
  author={Michael E. Tipping and Charles M. Bishop},
  journal={Journal of the Royal Statistical Society: Series B (Statistical Methodology)},
Principal component analysis (PCA) is a ubiquitous technique for data analysis and processing, but one which is not based on a probability model. We demonstrate how the principal axes of a set of observed data vectors may be determined through maximum likelihood estimation of parameters in a latent variable model that is closely related to factor analysis. We consider the properties of the associated likelihood function, giving an EM algorithm for estimating the principal subspace iteratively… 
Bayesian peA
This paper uses probabilistic reformulation as the basis for a Bayesian treatment of PCA to show that effective dimensionality of the latent space (equivalent to the number of retained principal components) can be determined automatically as part of the Bayesian inference procedure.
Latent Variable Models for Dimensionality Reduction
This paper provides a probabilistic interpretation of PCO and shows that this interpretation yields a maximum likelihood procedure for estimating the PCO parameters and also presents an iterative expectation-maximization algorithm for obtaining maximum likelihood estimates.
Interpretation of the Probabilistic Principal Components Analysis with Anisotropic Gaussian Distribution of Latent Variables
More general case in which the latent variables are independent with different variances is considered, and properties of the associated likelihood function are investigated.
Probabilistic Disjoint Principal Component Analysis
In this paper, disjoint principal component analysis model is extended in a maximum-likelihood framework to allow for inference on the model parameters and a coordinate ascent algorithm is proposed to estimate the models parameters.
Generalized probabilistic principal component analysis of correlated data
The generalized probabilistic principal component analysis is introduced to study the latent factor model for multiple correlated outcomes, where each factor is modeled by a Gaussian process and the closed-form maximum marginal likelihood estimator of the factor loadings and other parameters is provided.
A Generalization of Principal Components Analysis to the Exponential Family
This paper draws on ideas from the Exponential family, Generalized linear models, and Bregman distances to give a generalization of PCA to loss functions that it is argued are better suited to other data types.
Bayesian estimation of the number of principal components
Hidden Markov Bayesian Principal Component Analysis Hidden Markov Bayesian Principal Component Analysis
A temporal version of Probabilistic Principal Component Analysis is introduced by using a hidden Markov model in order to obtain optimized representations of observed data through time and a Bayesian treatment of the Principal Component model is introduced.
Sparse Kernel Principal Component Analysis
By approximating the covariance matrix in feature space by a reduced number of example vectors, using a maximum-likelihood approach, it is shown that a highly sparse form of kernel PCA can be obtained without loss of effectiveness.


An Expectation-Maximization Approach to Nonlinear Component Analysis
This work proposes an expectation-maximization approach for performing kernel principal component analysis and shows this to be a computationally efficient method, especially when the number of data points is large.
Abstract : The asymptotic distribution of the characteristic roots and (normalized) vectors of a sample covariance matrix is given when the observations are from a multivariate normal distribution
Nonlinear Component Analysis as a Kernel Eigenvalue Problem
A new method for performing a nonlinear form of principal component analysis by the use of integral operator kernel functions is proposed and experimental results on polynomial feature extraction for pattern recognition are presented.
EM algorithms for ML factor analysis
The details of EM algorithms for maximum likelihood factor analysis are presented for both the exploratory and confirmatory models. The algorithm is essentially the same for both cases and involves
Kernel principal component analysis for texture classification
Kernel principal component analysis (PCA) has recently been proposed as a nonlinear extension of PCA. The basic idea is to first map the input space into a feature space via a nonlinear map and then
Nonlinear kernel-based statistical pattern analysis
The Kernel version of Mahalanobis distance is introduced, which originates nonparametric models with unexpected and interesting properties, and a kernel version of the minimum squared error (MSE) linear discriminant function is proposed.
Maximum likelihood estimation and factor analysis
Fisher's method of maximum likelihood is applied to the problem of estimation in factor analysis, as initiated by Lawley, and found to lead to a generalization of the Eckart matrix approximation
Texture classification with kernel principal component analysis
Kernel principal component analysis (PCA) is presented as a mechanism for extracting textural information using the polynomial kernel, and supervised texture classification can be performed using a neural network.
An introduction to kernel-based learning algorithms
This paper provides an introduction to support vector machines, kernel Fisher discriminant analysis, and kernel principal component analysis, as examples for successful kernel-based learning methods.
Statistical Inference in Factor Analysis
This paper discusses some methods of factor analysis and considers some mathematical problems of the model, such as whether certain kinds of observed data determine the model uniquely, and treats the statistical problems of estimation and tests of certain hypotheses.