RGB-D Salient Object Detection with Cross-Modality Modulation and Selection

@article{Li2020RGBDSO,
  title={RGB-D Salient Object Detection with Cross-Modality Modulation and Selection},
  author={Chongyi Li and Runmin Cong and Yongri Piao and Q. Xu and Chen Change Loy},
  journal={ArXiv},
  year={2020},
  volume={abs/2007.07051}
}
We present an effective method to progressively integrate and refine the cross-modality complementarities for RGB-D salient object detection (SOD). The proposed network mainly solves two challenging issues: 1) how to effectively integrate the complementary information from RGB image and its corresponding depth map, and 2) how to adaptively select more saliency-related features. First, we propose a cross-modality feature modulation (cmFM) module to enhance feature representations by taking the… Expand
Specificity-preserving RGB-D Saliency Detection
  • Tao Zhou, Huazhu Fu, Geng Chen, Yi Zhou, Deng-Ping Fan, Ling Shao
  • Computer Science
  • ArXiv
  • 2021
RGB-D saliency detection has attracted increasing attention, due to its effectiveness and the fact that depth cues can now be conveniently captured. Existing works often focus on learning a sharedExpand
Deep RGB-D Saliency Detection with Depth-Sensitive Attention and Automatic Multi-Modal Fusion
TLDR
A depth-sensitive RGB feature modeling scheme using the depth-wise geometric prior of salient objects and an automatic architecture search approach for RGB-D SOD, which does well in finding out a feasible architecture from a specially designed multi-modal multi-scale search space. Expand
Cross-modality Discrepant Interaction Network for RGB-D Salient Object Detection
TLDR
A novel Cross-modality Discrepant Interaction Network (CDINet) for RGB-D SOD is proposed, which differentially models the dependence of two modalities according to the feature representations of different layers, and outperforms 15 state of the art methods both quantitatively and qualitatively. Expand
Hierarchical Alternate Interaction Network for RGB-D Salient Object Detection
TLDR
This article proposes a Hierarchical Alternate Interactions Network (HAINet) for RGB-D Salient Object Detection, which achieves competitive performance as compared with 19 relevant state-of-the-art methods, but also reaches a real-time processing speed of 43 fps on a single NVIDIA Titan X GPU. Expand
RGB-D Salient Object Detection With Ubiquitous Target Awareness
  • Yifan Zhao, Jiawei Zhao, Jia Li, Xiaowu Chen
  • Computer Science, Medicine
  • IEEE Transactions on Image Processing
  • 2021
Conventional RGB-D salient object detection methods aim to leverage depth as complementary information to find the salient regions in both modalities. However, the salient object detection resultsExpand
TriTransNet: RGB-D Salient Object Detection with a Triplet Transformer Embedding Network
TLDR
The proposed triplet transformer embedding network (TriTransNet) achieves the state-of-the-art performance in RGB-D salient object detection, and pushes the performance to a new level. Expand
Calibrated RGB-D Salient Object Detection
Complex backgrounds and similar appearances between objects and their surroundings are generally recognized as challenging scenarios in Salient Object Detection (SOD). This naturally leads to theExpand
RGB-D Salient Object Detection via 3D Convolutional Neural Networks
TLDR
The proposed model, named RD3D, aims at pre-fusion in the encoder stage and in-depth fusion in the decoder stage to effectively promote the full integration of RGB and depth streams and performs favorably against 14 state-of-the-art RGB-D SOD approaches in terms of four key evaluation metrics. Expand
Depth quality-aware selective saliency fusion for RGB-D image salient object detection
TLDR
The dense short-connections are used to enable a recursively interaction between the parallel sub networks, pursuing an optimal complementary status between multi-model deep features. Expand
Depth Quality-Inspired Feature Manipulation for Efficient RGB-D Salient Object Detection
TLDR
This work proposes a novel depth quality-inspired feature manipulation (DQFM) process, which is efficient itself and can serve as a gating mechanism for filtering depth features to greatly boost the accuracy of RGB-D salient object detection models. Expand
...
1
2
3
...

References

SHOWING 1-10 OF 50 REFERENCES
RGBD Salient Object Detection via Deep Fusion
TLDR
A new convolutional neural network is designed to automatically learn the interaction mechanism for RGBD salient object detection by adopting various flexible and interpretable saliency feature vectors as inputs and integrates a superpixel-based Laplacian propagation framework with the trained CNN to extract a spatially consistent saliency map. Expand
ASIF-Net: Attention Steered Interweave Fusion Network for RGB-D Salient Object Detection
TLDR
An attention steered interweave fusion network (ASIF-Net) is proposed to detect salient objects, which progressively integrates cross-modal and cross-level complementarity from the RGB image and corresponding depth map via steering of an attention mechanism. Expand
Discriminative Cross-Modal Transfer Learning and Densely Cross-Level Feedback Fusion for RGB-D Salient Object Detection
  • Hao Chen, Youfu Li, Dan Su
  • Computer Science, Medicine
  • IEEE Transactions on Cybernetics
  • 2020
TLDR
This article addresses two key issues in RGB-D salient object detection based on the convolutional neural network (CNN) with a densely cross-level feedback topology, in which the cross-modal complements are combined in each level and then densely fed back to all shallower layers for sufficient cross- level interactions. Expand
RGBD Salient Object Detection: A Benchmark and Algorithms
TLDR
A simple fusion framework that combines existing RGB-produced saliency with new depth-induced saliency and a specialized multi-stage RGBD model is proposed which takes account of both depth and appearance cues derived from low-level feature contrast, mid-level region grouping and high-level priors enhancement. Expand
Select, Supplement and Focus for RGB-D Saliency Detection
TLDR
A new framework for accurate RGB-D saliency detection taking account of local and global complementarities from two modalities is proposed by designing a complimentary interaction model discriminative enough to simultaneously select useful representation from RGB and depth data, and meanwhile to refine the object boundaries. Expand
CNNs-Based RGB-D Saliency Detection via Cross-View Transfer and Multiview Fusion.
TLDR
A novel framework based on convolutional neural networks (CNNs), which transfers the structure of the RGB-based deep neural network to be applicable for depth view and fuses the deep representations of both views automatically to obtain the final saliency map is proposed. Expand
Progressively Complementarity-Aware Fusion Network for RGB-D Salient Object Detection
  • H. Chen, Youfu Li
  • Computer Science
  • 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition
  • 2018
TLDR
A novel complementarity-aware fusion (CA-Fuse) module when adopting the Convolutional Neural Network (CNN) and the proposed RGB-D fusion network disambiguates both cross-modal and cross-level fusion processes and enables more sufficient fusion results. Expand
Three-Stream Attention-Aware Network for RGB-D Salient Object Detection
  • H. Chen, Youfu Li
  • Computer Science, Medicine
  • IEEE Transactions on Image Processing
  • 2019
TLDR
In the proposed architecture, a cross-modal distillation stream, accompanying the RGB-specific and depth-specific streams, is introduced to extract new RGB-D features in each level in the bottom–up path, and the channel-wise attention mechanism is innovatively introduced to the cross- modal cross-level fusion problem to adaptively select complementary feature maps from each modality in eachlevel. Expand
Going From RGB to RGBD Saliency: A Depth-Guided Transformation Model
TLDR
A novel depth-guided transformation model (DTM) going from RGB saliency to RGBD saliency is proposed and an optimization model is formulated to attain more consistent and accurate saliency results via an energy function, which integrates the unary data term, color smooth term, and depth consistency term. Expand
Contrast Prior and Fluid Pyramid Integration for RGBD Salient Object Detection
TLDR
Contrast prior is utilized, which used to be a dominant cue in none deep learning based SOD approaches, into CNNs-based architecture to enhance the depth information and is integrated with RGB features for SOD, using a novel fluid pyramid integration. Expand
...
1
2
3
4
5
...