Segmentation and semantic labelling of RGBD data with convolutional neural networks and surface fitting

  title={Segmentation and semantic labelling of RGBD data with convolutional neural networks and surface fitting},
  author={Giampaolo Pagnutti and L. Minto and Pietro Zanuttigh},
  journal={IET Comput. Vis.},
We present an approach for segmentation and semantic labelling of RGBD data exploiting together geometrical cues and deep learning techniques. An initial over-segmentation is performed using spectral clustering and a set of non-uniform rational B-spline surfaces is fitted on the extracted segments. Then a convolutional neural network (CNN) receives in input colour and geometry data together with surface fitting parameters. The network is made of nine convolutional stages followed by a softmax… 
Region Merging Driven by Deep Learning for RGB-D Segmentation and Labeling
This work proposes to exploit a convolutional neural network which takes a couple of segments as input and decides whether to join or not the segments, and fitted this idea into an existent iterative semantic segmentation scheme for RGB-D data.
2.5D Convolution for RGB-D Semantic Segmentation
2.5D convolution can effectively process spatial relations between pixels in a manner similar to 3D Convolution while still sampling pixels on 2D plane, and thus saves computational cost and can be seamlessly incorporated into pretrained CNNs.
Incremental Learning Techniques for Part-Based Semantic Segmentation
Semantic Image Segmentation is a computer vision task in which we label each single pixel in an image according to the semantic content. This task is of essential importance for a wide range of
Incremental and Multi-Task Learning Strategies for Coarse-To-Fine Semantic Segmentation
This work addresses the multi-level semantic segmentation task where a deep neural network is first trained to recognize an initial, coarse, set of a few classes and is adapted to segment and label new objects’ categories hierarchically derived from subdividing the classes of the initial set.
Deep learning for scene understanding with color and depth data
Three classical computer vision problems have been selected and novel approaches for their solution have been proposed in this work that both leverage the output of a deep Convolutional Neural Network (ConvNet) as well jointly exploit color and depth data to achieve competing results.
Robust Room Layout Estimation from a Single Image with Geometric Hints
  • Rui Deng, X. Chen
  • Computer Science
    2018 25th IEEE International Conference on Image Processing (ICIP)
  • 2018
A deep network that combines textures and geometric hints to predict the surface layout from a single image is proposed, demonstrating the robustness of the method on complex scenes.
Detection of similar objects and localizing on each using Depth Camera
  • R. S M, Anusha Akula
  • Computer Science
    2020 Second International Conference on Inventive Research in Computing Applications (ICIRCA)
  • 2020
The novelty in this work is in use of depth image from Intel 3D sensor in localization of each object, this does not require computation in image alignment of RGB image and depth image.
Gradient clustering algorithm based on deep learning aerial image detection
A parallel study based on deep learning on the gradient clustering algorithm of deep learning in aerial image processing using CUDA (Compute Unified Device Architecture) to perform large-scale parallel processing of aerial data.
Dynamic Point Cloud Geometry Compression via Patch-wise Polynomial Fitting
This work proposes a novel point cloud compression algorithm based on polynomial fitting of proper patches that achieves a noticeable improvement in the compression efficiency while maintaining point cloud quality.
Virtual Adversarial Training-Based Semisupervised Specific Emitter Identification
  • CunXiang Xie, LiMin Zhang, ZhaoGen Zhong
  • Wireless Communications and Mobile Computing
  • 2022
Deep learning is a new direction of research for specific emitter identification (SEI). Radio frequency (RF) fingerprints of the emitter signal are small and sensitive to noise. It is difficult to


Scene Segmentation Driven by Deep Learning and Surface Fitting
This paper proposes a joint color and depth segmentation scheme exploiting together geometrical clues and a learning stage and shows how the proposed method provides an accurate and reliable scene segmentation.
Joint Color and Depth Segmentation based on Region Merging and Surface Fitting
This paper proposes a region merging segmentation scheme that jointly exploits the two clues and uses a NURBS surface fitting scheme on the considered segments in order to understand if the regions candidate for the merging correspond to a single surface.
Scene segmentation from depth and color data driven by surface fitting
A novel segmentation scheme that exploits the joint usage of color and depth data together with 3D surface estimation scheme is proposed that allows to obtain an accurate and reliable scene segmentation.
Convolutional nets and watershed cuts for real-time semantic Labeling of RGBD videos
An efficient video segmentation approach that computes temporally consistent pixels in a causal manner is proposed, filling the need for causal and real-time applications.
Fusion of Geometry and Color Information for Scene Segmentation
A novel segmentation scheme where multidimensional vectors are used to jointly represent color and depth data and normalized cuts spectral clustering is applied to them in order to segment the scene.
Learning Hierarchical Features for Scene Labeling
A method that uses a multiscale convolutional network trained from raw pixels to extract dense feature vectors that encode regions of multiple sizes centered on each pixel, alleviates the need for engineered features, and produces a powerful representation that captures texture, shape, and contextual information.
Learning Rich Features from RGB-D Images for Object Detection and Segmentation
A new geocentric embedding is proposed for depth images that encodes height above ground and angle with gravity for each pixel in addition to the horizontal disparity to facilitate the use of perception in fields like robotics.
Learning Common and Specific Features for RGB-D Semantic Segmentation with Deconvolutional Networks
The proposed feature transformation network achieves competitive segmentation accuracy on NYU depth dataset V1 and V2 and takes advantage of deconvolutional networks which can predict pixel-wise class labels, and develops a new structure for deconvolved of multiple modalities.
Semantic Instance Labeling Leveraging Hierarchical Segmentation
This paper implements a higher level segmentation using a hierarchy of super pixels to obtain a better segmentation for training a classifier in indoor RGBD semantic labeling.
Region-Based Semantic Segmentation with End-to-End Training
We propose a novel method for semantic segmentation, the task of labeling each pixel in an image with a semantic class. Our method combines the advantages of the two main competing paradigms. Methods