Supervised Dimensionality Reduction via Distance Correlation Maximization

@article{Vepakomma2016SupervisedDR,
title={Supervised Dimensionality Reduction via Distance Correlation Maximization},
author={Praneeth Vepakomma and Chetan Tonde and A. Elgammal},
journal={ArXiv},
year={2016},
volume={abs/1601.00236}
}
• Published 3 January 2016
• Computer Science
• ArXiv
In our work, we propose a novel formulation for supervised dimensionality reduction based on a nonlinear dependency criterion called Statistical Distance Correlation, Szekely et. al. (2007). We propose an objective which is free of distributional assumptions on regression variables and regression model assumptions. Our proposed formulation is based on learning a low-dimensional feature representation $\mathbf{z}$, which maximizes the squared sum of Distance Correlations between low dimensional…

Figures and Tables from this paper

• Zhipeng Fan
• Computer Science
Journal of Statistical Computation and Simulation
• 2019
A supervised manifold learning method that makes use of the information of continuous dependent variables to distinguish intrinsic neighbourhood and extrinsic neighbourhood of data samples, and construct two graphs according to these two kinds of neighbourhoods.
• Computer Science
INFORMS J. Comput.
• 2021
We propose a novel supervised dimension reduction method, called supervised t-distributed stochastic neighbor embedding (St-SNE), which achieves dimension reduction by preserving the similarities of
• Computer Science
ArXiv
• 2020
This work proposes a deep dimension reduction (DDR) approach to achieving a good data representation with these characteristics for supervised learning, and formulate the ideal representation learning task as finding a nonlinear dimension reduction map that minimizes the sum of losses characterizing conditional independence and disentanglement.
• Computer Science
2020 25th International Conference on Pattern Recognition (ICPR)
• 2021
Experimental results confirm that the proposed method is effective in finding a discriminative representation of the features and outperforms several supervised embedding approaches in terms of classification performance.
• Computer Science
• 2017
In this paper we show that the negative sample distance covariance function is a quasi-concave set function of samples of random variables that are not statistically independent. We use these
• Computer Science
2018 International Joint Conference on Neural Networks (IJCNN)
• 2018
This paper elaborate on different properties of distance correlation to illustrate that maximizing it would be beneficial for supervised dimensionality reduction and described the general structure of the autoencoder used to maximize distance correlation.
• Computer Science, Mathematics
• 2019
The central limit theorems and the associated rates of convergence for a rescaled test statistic based on the bias-corrected distance correlation in high dimensions under some mild regularity conditions are established and reveal an interesting phenomenon of blessing of dimensionality for high-dimensional nonparametric inference with distance correlation.
• Computer Science
Journal of Applied Statistics
• 2020
A new filter approach to feature selection based on distance correlation is presented, which keeps the model-free advantage without any pre-specified parameters, and becomes more competitive on several datasets compared with some of the representative feature selection methods based on several classification models.
• Computer Science, Mathematics
ArXiv
• 2021
A novel differentially private method PrivateMail is presented for supervised manifold learning, the first of its kind to the authors' knowledge, and a novel private geometric embedding scheme is provided for this experimental use case.

References

SHOWING 1-10 OF 45 REFERENCES

• Computer Science
Neural Computation
• 2013
A novel sufficient dimension-reduction method using a squared-loss variant of mutual information as a dependency measure that is formulated as a minimum contrast estimator on parametric or nonparametric models and a natural gradient algorithm on the Grassmann manifold for sufficient subspace search.
• Computer Science
• 2014
It is proved that the proposed method is able to estimate the directions that achieve sufficient dimension reduction and has wide applicability without strong assumptions on the distributions or the type of variables, and needs only eigendecomposition for estimating the projection matrix.
• Computer Science
• 2016
This work introduces a novel approach to sufficient dimension-reduction problems using distance covariance, which keeps the model-free advantage without estimating link function, and finds that the method is very competitive and robust across a number of models.
• Computer Science
J. Mach. Learn. Res.
• 2003
A formal definition of the general continuous IB problem is given and an analytic solution for the optimal representation for the important case of multivariate Gaussian variables is obtained, in terms of the eigenvalue spectrum.
• Computer Science
• 2011
A novel distribution-free SDR method called sufficient component analysis (SCA) is proposed, which is computationally more efficient than existing methods and shown to compare favorably with existing dimension reduction approaches.
Abstract Modern advances in computing power have greatly widened scientists' scope in gathering and investigating information from many variables, information which might have been ignored in the
• Computer Science
• 2015
Methods to check goodness-of-fit for a given dimension reduction subspace are introduced to extend the so-called distance correlation to measure the conditional dependence relationship between the covariates and the response given a reduction sub space.
• Mathematics
• 2013
Variable selection is considered in the setting of supervised binary classification with functional data $\{X(t),\ t\in[0,1]\}$. By "variable selection" we mean any dimension-reduction method which
• Computer Science
• 2012
The authors offer a comprehensive introduction of various density ratio estimators including methods via density estimation, moment matching, probabilistic classification, density fitting, and density ratio fitting as well as describing how these can be applied to machine learning.
• Computer Science
ArXiv
• 2000
The variational principle provides a surprisingly rich framework for discussing a variety of problems in signal processing and learning, as will be described in detail elsewhere.