I Tried a Bunch of Things : the Dangers of Unexpected Overfitting in Classification

@inproceedings{Skocik2016ITA,
  title={I Tried a Bunch of Things : the Dangers of Unexpected Overfitting in Classification},
  author={Michael J. Skocik and John Collins and Chloe Callahan-Flintoft and Howard Bowman and Brad A Wyble},
  year={2016}
}
Machine learning is a powerful set of techniques that has enhanced the abilities of neuroscientists to interpret information collected through EEG, fMRI, MEG, and PET data. With these new techniques come new dangers of overfitting that are not well understood by the neuroscience community. In this article, we use Support Vector Machine (SVM) classifiers, and genetic algorithms to demonstrate the ease by which overfitting can occur, despite the use of cross validation. We demonstrate that… CONTINUE READING