Two-Layer Contractive Encodings with Shortcuts for Semi-supervised Learning

Abstract

Supervised training of multi-layer perceptrons (MLP) with only few labeled examples is prone to overfitting. Pretraining an MLP with unlabeled samples of the input distribution may achieve better generalization. Usually, pretraining is done in a layer-wise, greedy fashion which limits the complexity of the learnable features. To overcome this limitation, two-layer contractive encodings have been proposed recently—which pose a more difficult optimization problem, however. On the other hand, linear transformations of perceptrons have been proposed to make optimization of deep networks easier. In this paper, we propose to combine these two approaches. Experiments on handwritten digit recognition show the benefits of our combined approach to semi-supervised learning.

DOI: 10.1007/978-3-642-42054-2_56

Extracted Key Phrases

1 Figure or Table

Cite this paper

@inproceedings{Schulz2013TwoLayerCE, title={Two-Layer Contractive Encodings with Shortcuts for Semi-supervised Learning}, author={Hannes Schulz and Kyunghyun Cho and Tapani Raiko and Sven Behnke}, booktitle={ICONIP}, year={2013} }