Supervised Dimensionality Reduction via Distance Correlation Maximization

  title={Supervised Dimensionality Reduction via Distance Correlation Maximization},
  author={Praneeth Vepakomma and Chetan Tonde and A. Elgammal},
In our work, we propose a novel formulation for supervised dimensionality reduction based on a nonlinear dependency criterion called Statistical Distance Correlation, Szekely et. al. (2007). We propose an objective which is free of distributional assumptions on regression variables and regression model assumptions. Our proposed formulation is based on learning a low-dimensional feature representation $\mathbf{z}$, which maximizes the squared sum of Distance Correlations between low dimensional… 

Figures and Tables from this paper

A dimensionality reduction method of continuous dependent variables based supervised Laplacian eigenmaps

  • Zhipeng Fan
  • Computer Science
    Journal of Statistical Computation and Simulation
  • 2019
A supervised manifold learning method that makes use of the information of continuous dependent variables to distinguish intrinsic neighbourhood and extrinsic neighbourhood of data samples, and construct two graphs according to these two kinds of neighbourhoods.

Supervised t-Distributed Stochastic Neighbor Embedding for Data Visualization and Classification

We propose a novel supervised dimension reduction method, called supervised t-distributed stochastic neighbor embedding (St-SNE), which achieves dimension reduction by preserving the similarities of

Deep Dimension Reduction for Supervised Representation Learning

This work proposes a deep dimension reduction (DDR) approach to achieving a good data representation with these characteristics for supervised learning, and formulate the ideal representation learning task as finding a nonlinear dimension reduction map that minimizes the sum of losses characterizing conditional independence and disentanglement.

Supervised Feature Embedding for Classification by Learning Rank-based Neighborhoods

Experimental results confirm that the proposed method is effective in finding a discriminative representation of the features and outperforms several supervised embedding approaches in terms of classification performance.

Combinatorics of Distance Covariance: Inclusion-Minimal Maximizers of Quasi-Concave Set Functions for Diverse Variable Selection

In this paper we show that the negative sample distance covariance function is a quasi-concave set function of samples of random variables that are not statistically independent. We use these

Distance Correlation Autoencoder

This paper elaborate on different properties of distance correlation to illustrate that maximizing it would be beneficial for supervised dimensionality reduction and described the general structure of the autoencoder used to maximize distance correlation.

Asymptotic Distributions of High-Dimensional Nonparametric Inference with Distance Correlation

The central limit theorems and the associated rates of convergence for a rescaled test statistic based on the bias-corrected distance correlation in high dimensions under some mild regularity conditions are established and reveal an interesting phenomenon of blessing of dimensionality for high-dimensional nonparametric inference with distance correlation.

Feature selection based on distance correlation: a filter algorithm

A new filter approach to feature selection based on distance correlation is presented, which keeps the model-free advantage without any pre-specified parameters, and becomes more competitive on several datasets compared with some of the representative feature selection methods based on several classification models.

Differentially Private Supervised Manifold Learning with Applications like Private Image Retrieval

A novel differentially private method PrivateMail is presented for supervised manifold learning, the first of its kind to the authors' knowledge, and a novel private geometric embedding scheme is provided for this experimental use case.



Sufficient Dimension Reduction via Squared-Loss Mutual Information Estimation

A novel sufficient dimension-reduction method using a squared-loss variant of mutual information as a dependency measure that is formulated as a minimum contrast estimator on parametric or nonparametric models and a natural gradient algorithm on the Grassmann manifold for sufficient subspace search.

Gradient-Based Kernel Dimension Reduction for Regression

It is proved that the proposed method is able to estimate the directions that achieve sufficient dimension reduction and has wide applicability without strong assumptions on the distributions or the type of variables, and needs only eigendecomposition for estimating the projection matrix.

Sufficient Dimension Reduction via Distance Covariance

This work introduces a novel approach to sufficient dimension-reduction problems using distance covariance, which keeps the model-free advantage without estimating link function, and finds that the method is very competitive and robust across a number of models.

Information Bottleneck for Gaussian Variables

A formal definition of the general continuous IB problem is given and an analytic solution for the optimal representation for the important case of multivariate Gaussian variables is obtained, in terms of the eigenvalue spectrum.

Sufficient Component Analysis for Supervised Dimension Reduction

A novel distribution-free SDR method called sufficient component analysis (SCA) is proposed, which is computationally more efficient than existing methods and shown to compare favorably with existing dimension reduction approaches.

Sliced Inverse Regression for Dimension Reduction

Abstract Modern advances in computing power have greatly widened scientists' scope in gathering and investigating information from many variables, information which might have been ignored in the

Diagnostic studies in sufficient dimension reduction

Methods to check goodness-of-fit for a given dimension reduction subspace are introduced to extend the so-called distance correlation to measure the conditional dependence relationship between the covariates and the response given a reduction sub space.

Variable selection in functional data classification: a maxima-hunting proposal

Variable selection is considered in the setting of supervised binary classification with functional data $\{X(t),\ t\in[0,1]\}$. By "variable selection" we mean any dimension-reduction method which

Density Ratio Estimation in Machine Learning

The authors offer a comprehensive introduction of various density ratio estimators including methods via density estimation, moment matching, probabilistic classification, density fitting, and density ratio fitting as well as describing how these can be applied to machine learning.

The information bottleneck method

The variational principle provides a surprisingly rich framework for discussing a variety of problems in signal processing and learning, as will be described in detail elsewhere.