A non-asymptotic approach for model selection via penalization in high-dimensional mixture of experts models

  title={A non-asymptotic approach for model selection via penalization in high-dimensional mixture of experts models},
  author={TrungTin Nguyen and Hien Duy Nguyen and Faicel Chamroukhi and Florence Forbes},
  journal={Electronic Journal of Statistics},
: Mixture of experts (MoE) are a popular class of statistical and machine learning models that have gained attention over the years due to their flexibility and efficiency. In this work, we consider Gaussian- gated localized MoE (GLoME) and block-diagonal covariance localized MoE (BLoME) regression models to present nonlinear relationships in het- erogeneous data with potential hidden graph-structured interactions between high-dimensional predictors. These models pose difficult statistical… 
2 Citations

Summary statistics and discrepancy measures for approximate Bayesian computation via surrogate posteriors

This work introduces a preliminary learning step in which surrogate posteriors are built from finite Gaussian mixtures using an inverse regression approach that is seen as an extension of the semi-automatic ABC framework to a functional summary statistics setting and can also be used as an alternative to sample-based approaches.

Summary statistics and discrepancy measures for ABC via surrogate posteriors

A key ingredient in approximate Bayesian computation (ABC) procedures is the choice of a discrepancy that describes how different the simulated and observed data are, often based on a set of summary



ℓ1-penalization for mixture regression models

We consider a finite mixture of regressions (FMR) model for high-dimensional inhomogeneous data where the number of covariates may be much larger than sample size. We propose an ℓ1-penalized maximum

An l1-oracle inequality for the Lasso in finite mixture Gaussian regression models

The aim is to extend the l 1 -oracle inequality established by Massart and Meynet in the homogeneous Gaussian linear regression case, and to present a complementary result to Stadler et al.

High-dimensional regression with gaussian mixtures and partially-latent response variables

An inverse regression framework is proposed, which exchanges the roles of input and response, such that the low-dimensional variable becomes the regressor, and which is tractable and Experimental evidence is provided that the method outperforms several existing regression techniques.

An ℓ1-oracle inequality for the Lasso in multivariate finite mixture of multivariate Gaussian regression models

We consider a multivariate finite mixture of Gaussian regression models for high-dimensional data, where the number of covariates and the size of the response may be much larger than the sample size.

Hierarchical mixtures-of-experts for exponential family regression models: approximation and maximum

We consider hierarchical mixtures-of-experts (HME) models where exponential family regression models with generalized linear mean functions of the form ψ(α + x T β) are mixed. Here ψ(.) is the

Block-Diagonal Covariance Selection for High-Dimensional Gaussian Graphical Models

An application to a real gene expression dataset with a limited sample size is presented: the dimension reduction allows attention to be objectively focused on interactions among smaller subsets of genes, leading to a more parsimonious and interpretable modular network.

Model-based regression clustering for high-dimensional data: application to functional data

This article investigates the relationship between response and predictors arising from different subpopulations using finite mixture regression models and proposes two procedures to cluster observations according to the link between predictors and the response.

Data-driven Calibration of Penalties for Least-Squares Regression

A completely data-driven calibration algorithm for these parameters in the least-squares regression framework, without assuming a particular shape for the penalty, based on the concept of minimal penalty, recently introduced by Birge and Massart (2007).

Data-driven penalty calibration : a case study for Gaussian mixture model selection

The behavior of the data-driven criterion, a penalized likelihood criterion to select a Gaussian mixture model among a specific model collection, is highlighted on simulated datasets, a curve clustering example and a genomics application.

Inverse regression approach to robust nonlinear high-to-low dimensional mapping