Adaptive deconvolutional networks for mid and high level feature learning

@article{Zeiler2011AdaptiveDN,
  title={Adaptive deconvolutional networks for mid and high level feature learning},
  author={Matthew D. Zeiler and Graham W. Taylor and Rob Fergus},
  journal={2011 International Conference on Computer Vision},
  year={2011},
  pages={2018-2025}
}
We present a hierarchical model that learns image decompositions via alternating layers of convolutional sparse coding and max pooling. [...] Key Method To build our model we rely on a novel inference scheme that ensures each layer reconstructs the input, rather than just the output of the layer directly beneath, as is common with existing hierarchical approaches. This makes it possible to learn multiple layers of representation and we show models with 4 layers, trained on images from the Caltech-101 and 256…Expand
Detection guided deconvolutional network for hierarchical feature learning
TLDR
Experiments demonstrate that the proposed detection-guided hierarchical learning algorithm to train a deep deconvolutional network outperforms some recent hierarchical feature descriptors as well as classical hand-crafted features on the task of object recognition.
A Deconvolutional Competitive Algorithm for Building Sparse Hierarchical Representations
TLDR
A novel Deconvolutional Competitive Algorithm (DCA) is described, which explicitly learns non-redundant hierarchical representations by enabling competition both within and between sparse coding layers.
Learning Deep Representations, Embeddings and Codes from the Pixel Level of Natural and Medical Images
TLDR
This thesis considers the task of learning two specific types of image representations from standard size RGB images: a semi-supervised dense low-dimensional embedding and an unsupervised sparse binary code and introduces a new algorithm called the deep matching pursuit network (DMP) that efficiently learns features layer-by-layer from the pixel level without the need for backpropagation fine tuning.
Regularized Hierarchical Feature Learning with Non-negative Sparsity and Selectivity for Image Classification
TLDR
A hierarchical deconvolutional network with two biologically inspired properties incorporated, i.e., non-negative sparsity and selectivity is proposed, to enhance the ability of the generative model to tackle discriminative computer vision tasks.
Image salient object detection with refined deep features via convolution neural network
TLDR
A convolutional neural network model to learn high-level features for saliency detection is proposed and deep belief network classifier is added at the end of this model to classify salient and nonsalient regions.
Modeling Neuron Selectivity Over Simple Midlevel Features for Image Classification
TLDR
An efficient unsupervised midlevel feature learning approach (MidFea), which only involves simple operations, such as k-means clustering, convolution, pooling, vector quantization, and random projection, and shows this simple feature can also achieve good performance in traditional classification task.
Stacked Convolutional Denoising Auto-Encoders for Feature Representation
TLDR
An unsupervised deep network, called the stacked convolutional denoising auto-encoders, which can map images to hierarchical representations without any label information is proposed, which demonstrates superior classification performance to state-of-the-art un supervised networks.
Learning Mid-Level Features and Modeling Neuron Selectivity for Image Classification
TLDR
This paper proposes to model the neuron selectivity (NS) principle by building an additional layer over the mid-level features before feeding the features into the classifier, and shows that the NS-layer learns category-specific neurons with both bottom-up inference and top-down analysis, and thus supports fast inference for a query image.
Differentiable Pooling for Hierarchical Feature Learning
TLDR
A parametric form of pooling, based on a Gaussian, which can be optimized alongside the features in a single global objective function, which provides a what/where decomposition of the input signal.
Learning and Transferring Mid-level Image Representations Using Convolutional Neural Networks
TLDR
This work designs a method to reuse layers trained on the ImageNet dataset to compute mid-level image representation for images in the PASCAL VOC dataset, and shows that despite differences in image statistics and tasks in the two datasets, the transferred representation leads to significantly improved results for object and action classification.
...
1
2
3
4
5
...

References

SHOWING 1-10 OF 32 REFERENCES
Deconvolutional networks
TLDR
This work presents a learning framework where features that capture these mid-level cues spontaneously emerge from image data, based on the convolutional decomposition of images under a spar-sity constraint and is totally unsupervised.
Learning Convolutional Feature Hierarchies for Visual Recognition
TLDR
This work proposes an unsupervised method for learning multi-stage hierarchies of sparse convolutional features and trains an efficient feed-forward encoder that predicts quasi-sparse features from the input.
Convolutional deep belief networks for scalable unsupervised learning of hierarchical representations
TLDR
The convolutional deep belief network is presented, a hierarchical generative model which scales to realistic image sizes and is translation-invariant and supports efficient bottom-up and top-down probabilistic inference.
Unsupervised Learning of Invariant Feature Hierarchies with Applications to Object Recognition
TLDR
An unsupervised method for learning a hierarchy of sparse feature detectors that are invariant to small shifts and distortions that alleviates the over-parameterization problems that plague purely supervised learning procedures, and yields good performance with very few labeled training samples.
Deep Learning with Hierarchical Convolutional Factor Analysis
Unsupervised multilayered (“deep”) models are considered for imagery. The model is represented using a hierarchical convolutional factor-analysis construction, with sparse factor loadings and scores.
Learning mid-level features for recognition
TLDR
This work seeks to establish the relative importance of each step of mid-level feature extraction through a comprehensive cross evaluation of several types of coding modules and pooling schemes and shows how to improve the best performing coding scheme by learning a supervised discriminative dictionary for sparse coding.
Are sparse representations really relevant for image classification?
TLDR
In experiments on the CIFAR-IO and on the Caltech-101 datasets, enforcing sparsity constraints actually does not improve recognition performance, which has an important practical impact in image descriptor design, as enforcing these constraints can have a heavy computational cost.
Is Sparsity Really Relevant for Image Classification
TLDR
In these experiments, enforcing sparsity constraints is not required at run-time, since no performance improvement has been reported by indiscriminatively sparsifying the descriptors, and is of practical interest, as enforcing these constraints can have a heavy computational cost.
What is the best multi-stage architecture for object recognition?
TLDR
It is shown that using non-linearities that include rectification and local contrast normalization is the single most important ingredient for good accuracy on object recognition benchmarks and that two stages of feature extraction yield better accuracy than one.
Object recognition with features inspired by visual cortex
TLDR
The performance of the approach constitutes a suggestive plausibility proof for a class of feedforward models of object recognition in cortex and exhibits excellent recognition performance and outperforms several state-of-the-art systems on a variety of image datasets including many different object categories.
...
1
2
3
4
...