Multimodal Recurrent Neural Networks With Information Transfer Layers for Indoor Scene Labeling

  title={Multimodal Recurrent Neural Networks With Information Transfer Layers for Indoor Scene Labeling},
  author={Abrar H. Abdulnabi and B. Shuai and Z. Zuo and Lap-Pui Chau and G. Wang},
  journal={IEEE Transactions on Multimedia},
  • Abrar H. Abdulnabi, B. Shuai, +2 authors G. Wang
  • Published 2018
  • Computer Science
  • IEEE Transactions on Multimedia
  • This paper proposes a new method called multimodal recurrent neural networks (RNNs) for RGB-D scene semantic segmentation. [...] Key Method Each RNN model learns its representations from its own previous hidden states and transferred patterns from the other RNNs previous hidden states; thus, both model-specific and cross-modality features are retained. We exploit the structure of quad-directional 2D-RNNs to model the short- and long-range contextual information in the 2D input image.Expand Abstract
    9 Citations
    Optimizing Features Quality: A Normalized Covariance Fusion Framework for Skeleton Action Recognition
    • PDF
    DeepFacade: A Deep Learning Approach to Facade Parsing With Symmetric Loss
    • 1
    Multi-Modal Recurrent Attention Networks for Facial Expression Recognition
    • 2
    • PDF
    RGB-D Semantic Segmentation: A Review
    • Y. Hu, Z. Chen, W. Lin
    • Computer Science
    • 2018 IEEE International Conference on Multimedia & Expo Workshops (ICMEW)
    • 2018
    • 3
    Weakly Supervised 3D Object Detection from Point Clouds
    • 2
    • PDF
    RiFeGAN: Rich Feature Generation for Text-to-Image Synthesis From Prior Knowledge
    • 4
    • PDF
    Multimodal Deep Learning in Semantic Image Segmentation: A Review
    • 2
    A Survey on Deep Learning for Multimodal Data Fusion
    • 16


    Scene labeling with LSTM recurrent neural networks
    • 264
    • PDF
    Multi-modal Unsupervised Feature Learning for RGB-D Scene Labeling
    • 50
    • PDF
    Recurrent Convolutional Neural Networks for Scene Labeling
    • 588
    • PDF
    Convolutional recurrent neural networks: Learning spatial dependencies for image representation
    • Zhen Zuo, B. Shuai, +4 authors Yushi Chen
    • Computer Science
    • 2015 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW)
    • 2015
    • 107
    • PDF
    Learning depth-sensitive conditional random fields for semantic segmentation of RGB-D images
    • A. Müller, Sven Behnke
    • Computer Science
    • 2014 IEEE International Conference on Robotics and Automation (ICRA)
    • 2014
    • 74
    • PDF
    Episodic CAMN: Contextual Attention-Based Memory Networks with Iterative Feedback for Scene Labeling
    • 12
    • PDF
    Look Closer to See Better: Recurrent Attention Convolutional Neural Network for Fine-Grained Image Recognition
    • 506
    • PDF
    RGB-(D) scene labeling: Features and algorithms
    • X. Ren, L. Bo, D. Fox
    • Computer Science
    • 2012 IEEE Conference on Computer Vision and Pattern Recognition
    • 2012
    • 419
    • PDF
    Fully Convolutional Networks for Semantic Segmentation
    • 9,049
    • Highly Influential
    • PDF
    Perceptual Organization and Recognition of Indoor Scenes from RGB-D Images
    • 487
    • PDF