Corpus ID: 16231549

Denoising autoencoder with modulated lateral connections learns invariant representations of natural images

@article{Rasmus2015DenoisingAW,
  title={Denoising autoencoder with modulated lateral connections learns invariant representations of natural images},
  author={Antti Rasmus and T. Raiko and H. Valpola},
  journal={CoRR},
  year={2015},
  volume={abs/1412.7210}
}
  • Antti Rasmus, T. Raiko, H. Valpola
  • Published 2015
  • Computer Science, Mathematics
  • CoRR
  • Suitable lateral connections between encoder and decoder are shown to allow higher layers of a denoising autoencoder (dAE) to focus on invariant representations. In regular autoencoders, detailed information needs to be carried through the highest layers but lateral connections from encoder to decoder relieve this pressure. It is shown that abstract invariant features can be translated to detailed reconstructions when invariant features are allowed to modulate the strength of the lateral… CONTINUE READING
    17 Citations

    Figures, Tables, and Topics from this paper

    Lateral Connections in Denoising Autoencoders Support Supervised Learning
    • 18
    • PDF
    A study on the similarities of Deep Belief Networks and Stacked Autoencoders
    • 4
    Theta-RBM: Unfactored Gated Restricted Boltzmann Machine for Rotation-Invariant Representations
    • 1
    • Highly Influenced
    • PDF
    BLAN: Bi-directional ladder attentive network for facial attribute prediction
    • PDF
    Unsupervised Rotation Factorization in Restricted Boltzmann Machines
    • 2
    • Highly Influenced
    • PDF
    Semi-Supervised Learning with Ladder Network
    • 94
    • PDF
    Semi-supervised Learning with Ladder Networks
    • 785
    • PDF
    Ladder Networks: Learning under Massive Label Deficit
    • 2
    • PDF

    References

    SHOWING 1-10 OF 25 REFERENCES
    From neural PCA to deep unsupervised learning
    • 136
    • PDF
    Extracting and composing robust features with denoising autoencoders
    • 4,354
    • PDF
    Deep Learning Made Easier by Linear Transformations in Perceptrons
    • 163
    • PDF
    Generalized Denoising Auto-Encoders as Generative Models
    • 370
    • PDF
    Measuring Invariances in Deep Networks
    • 370
    • PDF
    Gradient-based learning of higher-order image features
    • R. Memisevic
    • Computer Science
    • 2011 International Conference on Computer Vision
    • 2011
    • 65
    • PDF
    ImageNet classification with deep convolutional neural networks
    • 59,714
    • PDF
    Emergence of Phase- and Shift-Invariant Features by Decomposition of Natural Images into Independent Feature Subspaces
    • 596
    • PDF
    Learning Multiple Layers of Features from Tiny Images
    • 10,524
    • Highly Influential
    • PDF