# Supervised Dimensionality Reduction via Distance Correlation Maximization

@article{Vepakomma2016SupervisedDR, title={Supervised Dimensionality Reduction via Distance Correlation Maximization}, author={Praneeth Vepakomma and Chetan Tonde and A. Elgammal}, journal={ArXiv}, year={2016}, volume={abs/1601.00236} }

In our work, we propose a novel formulation for supervised dimensionality reduction based on a nonlinear dependency criterion called Statistical Distance Correlation, Szekely et. al. (2007). We propose an objective which is free of distributional assumptions on regression variables and regression model assumptions. Our proposed formulation is based on learning a low-dimensional feature representation $\mathbf{z}$, which maximizes the squared sum of Distance Correlations between low dimensional…

## 32 Citations

### A dimensionality reduction method of continuous dependent variables based supervised Laplacian eigenmaps

- Computer ScienceJournal of Statistical Computation and Simulation
- 2019

A supervised manifold learning method that makes use of the information of continuous dependent variables to distinguish intrinsic neighbourhood and extrinsic neighbourhood of data samples, and construct two graphs according to these two kinds of neighbourhoods.

### Supervised t-Distributed Stochastic Neighbor Embedding for Data Visualization and Classification

- Computer ScienceINFORMS J. Comput.
- 2021

We propose a novel supervised dimension reduction method, called supervised t-distributed stochastic neighbor embedding (St-SNE), which achieves dimension reduction by preserving the similarities of…

### Deep Dimension Reduction for Supervised Representation Learning

- Computer ScienceArXiv
- 2020

This work proposes a deep dimension reduction (DDR) approach to achieving a good data representation with these characteristics for supervised learning, and formulate the ideal representation learning task as finding a nonlinear dimension reduction map that minimizes the sum of losses characterizing conditional independence and disentanglement.

### Supervised discriminative dimensionality reduction by learning multiple transformation operators

- Computer ScienceExpert Syst. Appl.
- 2021

### Supervised Feature Embedding for Classification by Learning Rank-based Neighborhoods

- Computer Science2020 25th International Conference on Pattern Recognition (ICPR)
- 2021

Experimental results confirm that the proposed method is effective in finding a discriminative representation of the features and outperforms several supervised embedding approaches in terms of classification performance.

### Combinatorics of Distance Covariance: Inclusion-Minimal Maximizers of Quasi-Concave Set Functions for Diverse Variable Selection

- Computer Science
- 2017

In this paper we show that the negative sample distance covariance function is a quasi-concave set function of samples of random variables that are not statistically independent. We use these…

### Distance Correlation Autoencoder

- Computer Science2018 International Joint Conference on Neural Networks (IJCNN)
- 2018

This paper elaborate on different properties of distance correlation to illustrate that maximizing it would be beneficial for supervised dimensionality reduction and described the general structure of the autoencoder used to maximize distance correlation.

### Asymptotic Distributions of High-Dimensional Nonparametric Inference with Distance Correlation

- Computer Science, Mathematics
- 2019

The central limit theorems and the associated rates of convergence for a rescaled test statistic based on the bias-corrected distance correlation in high dimensions under some mild regularity conditions are established and reveal an interesting phenomenon of blessing of dimensionality for high-dimensional nonparametric inference with distance correlation.

### Feature selection based on distance correlation: a filter algorithm

- Computer ScienceJournal of Applied Statistics
- 2020

A new filter approach to feature selection based on distance correlation is presented, which keeps the model-free advantage without any pre-specified parameters, and becomes more competitive on several datasets compared with some of the representative feature selection methods based on several classification models.

### Differentially Private Supervised Manifold Learning with Applications like Private Image Retrieval

- Computer Science, MathematicsArXiv
- 2021

A novel differentially private method PrivateMail is presented for supervised manifold learning, the first of its kind to the authors' knowledge, and a novel private geometric embedding scheme is provided for this experimental use case.

## References

SHOWING 1-10 OF 45 REFERENCES

### Sufficient Dimension Reduction via Squared-Loss Mutual Information Estimation

- Computer ScienceNeural Computation
- 2013

A novel sufficient dimension-reduction method using a squared-loss variant of mutual information as a dependency measure that is formulated as a minimum contrast estimator on parametric or nonparametric models and a natural gradient algorithm on the Grassmann manifold for sufficient subspace search.

### Gradient-Based Kernel Dimension Reduction for Regression

- Computer Science
- 2014

It is proved that the proposed method is able to estimate the directions that achieve sufficient dimension reduction and has wide applicability without strong assumptions on the distributions or the type of variables, and needs only eigendecomposition for estimating the projection matrix.

### Sufficient Dimension Reduction via Distance Covariance

- Computer Science
- 2016

This work introduces a novel approach to sufficient dimension-reduction problems using distance covariance, which keeps the model-free advantage without estimating link function, and finds that the method is very competitive and robust across a number of models.

### Information Bottleneck for Gaussian Variables

- Computer ScienceJ. Mach. Learn. Res.
- 2003

A formal definition of the general continuous IB problem is given and an analytic solution for the optimal representation for the important case of multivariate Gaussian variables is obtained, in terms of the eigenvalue spectrum.

### Sufficient Component Analysis for Supervised Dimension Reduction

- Computer Science
- 2011

A novel distribution-free SDR method called sufficient component analysis (SCA) is proposed, which is computationally more efficient than existing methods and shown to compare favorably with existing dimension reduction approaches.

### Sliced Inverse Regression for Dimension Reduction

- Mathematics
- 1991

Abstract Modern advances in computing power have greatly widened scientists' scope in gathering and investigating information from many variables, information which might have been ignored in the…

### Diagnostic studies in sufficient dimension reduction

- Computer Science
- 2015

Methods to check goodness-of-fit for a given dimension reduction subspace are introduced to extend the so-called distance correlation to measure the conditional dependence relationship between the covariates and the response given a reduction sub space.

### Variable selection in functional data classification: a maxima-hunting proposal

- Mathematics
- 2013

Variable selection is considered in the setting of supervised binary classification with functional data $\{X(t),\ t\in[0,1]\}$. By "variable selection" we mean any dimension-reduction method which…

### Density Ratio Estimation in Machine Learning

- Computer Science
- 2012

The authors offer a comprehensive introduction of various density ratio estimators including methods via density estimation, moment matching, probabilistic classification, density fitting, and density ratio fitting as well as describing how these can be applied to machine learning.

### The information bottleneck method

- Computer ScienceArXiv
- 2000

The variational principle provides a surprisingly rich framework for discussing a variety of problems in signal processing and learning, as will be described in detail elsewhere.