Feature subset selection using a new definition of classifiability


The performance of most practical classifiers improves when correlated or irrelevant features are removed. Machine based classification is thus often preceded by subset selection – a procedure which identifies relevant features of a high dimensional data set. At present, the most widely used subset selection technique is the so-called “wrapper” approach in which a search algorithm is used to identify candidate subsets and the actual classifier is used as a “black box” to evaluate the fitness of the subset. Fitness evaluation of the subset however requires cross-validation or other resampling based procedure for error estimation necessitating the construction of a large number of classifiers for each subset. This significant computational burden makes the wrapper approach impractical when a large number of features are present. In this paper, we present an approach to subset selection based on a novel definition of the classifiability of a given data. The classifiability measure we propose characterizes the relative ease with which some labeled data can be classified. We use this definition of classifiability to systematically add the feature which leads to the most increase in classifiability. The proposed approach does not require the construction of classifiers at each step and therefore does not suffer from as high a computational burden as a wrapper approach. Our results over several different data sets indicate that the results obtained are at least as good as that obtained with the wrapper approach.

DOI: 10.1016/S0167-8655(02)00303-3

Extracted Key Phrases

7 Figures and Tables


Citations per Year

81 Citations

Semantic Scholar estimates that this publication has 81 citations based on the available data.

See our FAQ for additional information.

Cite this paper

@article{Dong2003FeatureSS, title={Feature subset selection using a new definition of classifiability}, author={Ming Dong and Ravi Kothari}, journal={Pattern Recognition Letters}, year={2003}, volume={24}, pages={1215-1225} }