An automatic robust Bayesian approach to principal component regression

@article{Gagnon2020AnAR,
  title={An automatic robust Bayesian approach to principal component regression},
  author={Philippe Gagnon and Mylene B'edard and Alain Desgagn'e},
  journal={Journal of Applied Statistics},
  year={2020},
  volume={48},
  pages={84 - 104}
}
Principal component regression uses principal components (PCs) as regressors. It is particularly useful in prediction settings with high-dimensional covariates. The existing literature treating of Bayesian approaches is relatively sparse. We introduce a Bayesian approach that is robust to outliers in both the dependent variable and the covariates. Outliers can be thought of as observations that are not in line with the general trend. The proposed approach automatically penalises these… 
Robustness against conflicting prior information in regression
Including prior information about model parameters is a fundamental step of any Bayesian statistical analysis. It is viewed positively by some as it allows, among others, to quantitatively
Optimal scaling of random walk Metropolis algorithms using Bayesian large-sample asymptotics
TLDR
The results highlight that the correlation structure has to be accounted for to avoid performance deterioration if that is not the case, while justifying the use of a natural (asymptotically exact) approximation to the correlation matrix that can be employed for the very first algorithm run.
A step further towards automatic and efficient reversible jump algorithms
TLDR
This paper builds on the approach of Zanella (2019) for discrete spaces to incorporate information about neighbouring models and relies on approximations to posterior model probabilities that are asymptotically exact, as the sample size increases.
Informed reversible jump algorithms
TLDR
This paper builds on the approach of Zanella (2020) for discrete spaces to incorporate information about neighbouring models and relies on approximations to posterior model probabilities that are asymptotically exact, proving that, in some scenarios, the samplers behave like those that use the exact model probabilities and sample from the parameter distributions, in the large sample regime.
Theoretical properties of Bayesian Student-$t$ linear regression
TLDR
The trade-off between robustness and efficiency controlled through the degrees of freedom is precisely characterized (at least asymptotically) in the Student-t linear regression scenario.
An asymptotic Peskun ordering and its application to lifted samplers
A Peskun ordering between two samplers, implying a dominance of one over the other, is known among the Markov chain Monte Carlo community for being a remarkably strong result, but it is also known
Nonreversible Jump Algorithms for Bayesian Nested Model Selection
TLDR
By lifting this model indicator variable, a nonreversible version of the popular reversible jump algorithms, this simple algorithmic modification provides samplers which can empirically outperform their reversible counterparts at no extra computational cost.

References

SHOWING 1-10 OF 49 REFERENCES
A New Bayesian Approach to Robustness Against Outliers in Linear Regression
TLDR
This paper proposes a model with super heavy-tailed errors, and proves that it is wholly robust, meaning that the impact of outliers gradually vanishes as they move further and further away form the general trend.
On Bayesian principal component analysis
Bayesian Robustness to Outliers in Linear Regression
Whole robustness is an appealing attribute to look for in statistical models. It implies that the impact of outliers, defined here as the observations that are not in line with the general trend,
Bayesian principal component regression with data-driven component selection
Principal component regression (PCR) has two steps: estimating the principal components and performing the regression using these components. These steps generally are performed sequentially. In PCR,
Bayesian robustness to outliers in linear regression and ratio estimation
Whole robustness is a nice property to have for statistical models. It implies that the impact of outliers gradually decreases to nothing as they converge towards plus or minus infinity. So far, the
Outlier Models and Prior Distributions in Bayesian Linear Regression
SUMMARY Bayesian inference in regression models is considered using heavy-tailed error distri- butions to accommodate outliers. The particular class of distributions that can be con- structed as
Bayesian Mode Regression
TLDR
A parametric Bayesian model is developed by employing a likelihood function that is based on a mode uniform distribution and it is shown that irrespective of the original distribution of the data, the use of this special uniform distribution is a very natural and effective way for Bayesian mode regression.
Bayesian factor regression models in the''large p
TLDR
Bayesian factor regression models with many explanatory variables are discussed, and sparse latent factor models are introduced to induce sparsity in factor loadings matrices to provide a novel approach to variable selection with very many predictors.
Bayesian Robust PCA for Incomplete Data
TLDR
A probabilistic model for robust principal component analysis (PCA) in which the observation noise is modelled by Student-t distributions that are independent for different data dimensions that can yield more accurate reconstructions of missing values.
A robust PCR method for high‐dimensional regressors
We consider the multivariate calibration model which assumes that the concentrations of several constituents of a sample are linearly related to its spectrum. Principal component regression (PCR) is
...
1
2
3
4
5
...