Corpus ID: 24835629

CortexNet: a Generic Network Family for Robust Visual Temporal Representations

@article{Canziani2017CortexNetAG,
  title={CortexNet: a Generic Network Family for Robust Visual Temporal Representations},
  author={A. Canziani and Eugenio Culurciello},
  journal={ArXiv},
  year={2017},
  volume={abs/1706.02735}
}
In the past five years we have observed the rise of incredibly well performing feed-forward neural networks trained supervisedly for vision related tasks. These models have achieved super-human performance on object recognition, localisation, and detection in still images. However, there is a need to identify the best strategy to employ these networks with temporal visual inputs and obtain a robust and stable representation of video data. Inspired by the human visual system, we propose a deep… Expand
Deep Recurrent Neural Network Reveals a Hierarchy of Process Memory during Dynamic Natural Vision
TLDR
The RNN better predicted cortical responses to natural movie stimuli than the CNN, at all visual areas especially those along the dorsal stream, and revealed a cortical hierarchy of temporal receptive window, dynamics of process memory, and spatiotemporal representations. Expand
Deep recurrent neural network reveals a hierarchy of process memory during dynamic natural vision
TLDR
The RNN better predicted cortical responses to natural movie stimuli than the CNN, at all visual areas, especially those along the dorsal stream, and revealed a cortical hierarchy of temporal receptive window, dynamics of process memory, and spatiotemporal representations. Expand
Integrating Motion into Vision Models for Better Visual Prediction
TLDR
An improved vision system that learns a model of its environment using a self-supervised, predictive learning method based on the Predictive Vision Model is demonstrated and the integration of camera motion into the predictive learning system is detailed and improved visual prediction and saccadic behavior is shown. Expand
Deep Predictive Coding Network for Object Recognition
TLDR
Deep predictive coding networks (PCN) reuses a single architecture to recursively run bottom-up and top-down process, enabling an increasingly longer cascade of non-linear transformation. Expand
Instance-level Object Recognition Using Deep Temporal Coherence
TLDR
This paper evaluates the use of temporal filters such as Cumulative Moving Average and a machine learning approach using Recurrent Neural Networks for this task of instance object recognition and shows that by exploiting temporal coherence, models trained with a few data points perform comparably to when the whole dataset is available. Expand
Deep in-situ learning for object recognition
TLDR
A scalable, robust, and real-time system based on Convolutional Neural Networks that learn discriminative features from images depicting objects from an egocentric point of view is aimed at. Expand
ToyArchitecture: Unsupervised learning of interpretable models of the environment
TLDR
This paper presents a novel, purposely simple, and interpretable hierarchical architecture that incorporates the unsupervised learning of a model of the environment, learning the influence of one’s own actions, model-based reinforcement learning, hierarchical planning, and symbolic/sub-symbolic integration in general. Expand
Recurrent Ladder Networks
TLDR
The recurrent Ladder is able to handle a wide variety of complex learning tasks that benefit from iterative inference and temporal modeling and shows close-to-optimal results on temporal modeling of video data, competitive results on music modeling, and improved perceptual grouping based on higher order abstractions. Expand
ToyArchitecture: Unsupervised Learning of Interpretable Models of the World
TLDR
This work presents a novel, purposely simple, and interpretable hierarchical architecture which combines multiple different mechanisms into one system: unsupervised learning of a model of the world, learning the influence of one’s own actions on theworld, model-based reinforcement learning, hierarchical planning and plan execution, and symbolic/sub-symbolic integration in general. Expand
Variational Predictive Routing with Nested Subjective Timescales
TLDR
This work presents Variational Predictive Routing (VPR) – a neural probabilistic inference system that organizes latent representations of video features in a temporal hierarchy, based on their rates of change, thus modeling continuous data as a hierarchical renewal process. Expand
...
1
2
...

References

SHOWING 1-10 OF 21 REFERENCES
Visual attention with deep neural networks
TLDR
This article presents a novel fully-learnt unbiassed biologically plausible algorithm for computing both feature based and proto-object saliency maps, using a deep convolutional neural network simply trained on a single-class classification task, by unveiling its internal attentional apparatus. Expand
Unsupervised Learning of Visual Representations Using Videos
  • X. Wang, A. Gupta
  • Computer Science
  • 2015 IEEE International Conference on Computer Vision (ICCV)
  • 2015
TLDR
A simple yet surprisingly powerful approach for unsupervised learning of CNN that uses hundreds of thousands of unlabeled videos from the web to learn visual representations and designs a Siamese-triplet network with a ranking loss function to train this CNN representation. Expand
Learning Features by Watching Objects Move
TLDR
Inspired by the human visual system, low-level motion-based grouping cues can be used to learn an effective visual representation that significantly outperforms previous unsupervised approaches across multiple settings, especially when training data for the target task is scarce. Expand
Deep Predictive Coding Networks for Video Prediction and Unsupervised Learning
TLDR
The results suggest that prediction represents a powerful framework for unsupervised learning, allowing for implicit learning of object and scene structure. Expand
Dynamic Model of Visual Recognition Predicts Neural Response Properties in the Visual Cortex
TLDR
A hierarchical network model of visual recognition that explains experimental observations regarding neural responses in both free viewing and fixating conditions by using a form of the extended Kalman filter as given by the minimum description length (MDL) principle is described. Expand
Deep Predictive Coding Networks
TLDR
Deep predictive coding networks is proposed, a hierarchical generative model that empirically alters priors on the latent representations in a dynamic and context-sensitive manner that captures the temporal dependencies in time-varying signals and uses top-down information to modulate the representation in lower layers. Expand
Deep neural networks are easily fooled: High confidence predictions for unrecognizable images
TLDR
This work takes convolutional neural networks trained to perform well on either the ImageNet or MNIST datasets and finds images with evolutionary algorithms or gradient ascent that DNNs label with high confidence as belonging to each dataset class, and produces fooling images, which are then used to raise questions about the generality of DNN computer vision. Expand
Anticipating Visual Representations from Unlabeled Video
TLDR
This work presents a framework that capitalizes on temporal structure in unlabeled video to learn to anticipate human actions and objects and applies recognition algorithms on the authors' predicted representation to anticipate objects and actions. Expand
Unsupervised Learning of Video Representations using LSTMs
TLDR
This work uses Long Short Term Memory networks to learn representations of video sequences and evaluates the representations by finetuning them for a supervised learning problem - human action recognition on the UCF-101 and HMDB-51 datasets. Expand
Spatio-temporal video autoencoder with differentiable memory
TLDR
One direct application of the proposed framework in weakly-supervised semantic segmentation of videos through label propagation using optical flow is presented, using as temporal decoder a robust optical flow prediction module together with an image sampler serving as built-in feedback loop. Expand
...
1
2
3
...