Bayesian SegNet: Model Uncertainty in Deep Convolutional Encoder-Decoder Architectures for Scene Understanding

@article{Kendall2017BayesianSM,
  title={Bayesian SegNet: Model Uncertainty in Deep Convolutional Encoder-Decoder Architectures for Scene Understanding},
  author={Alex Kendall and Vijay Badrinarayanan and Roberto Cipolla},
  journal={ArXiv},
  year={2017},
  volume={abs/1511.02680}
}
We present a deep learning framework for probabilistic pixel-wise semantic segmentation, which we term Bayesian SegNet. [] Key Method We achieve this by Monte Carlo sampling with dropout at test time to generate a posterior distribution of pixel class labels. In addition, we show that modelling uncertainty improves segmentation performance by 2-3% across a number of state of the art architectures such as SegNet, FCN and Dilation Network, with no additional parametrisation. We also observe a significant…

Figures and Tables from this paper

Deep convolutional encoder-decoder network with model uncertainty for semantic segmentation
  • S. Isobe, Shuichi Arai
  • Computer Science
    2017 IEEE International Conference on INnovations in Intelligent SysTems and Applications (INISTA)
  • 2017
TLDR
The improvement of certainty for semantic segmentation in road scene understanding with model uncertainty is achieved by the method that rejects the uncertain region and classifies it as an unknown object using the model uncertainty.
Inference with model uncertainty on indoor scene for semantic segmentation
  • S. Isobe, Shuichi Arai
  • Computer Science
    2017 IEEE Global Conference on Signal and Information Processing (GlobalSIP)
  • 2017
TLDR
This paper presents a novel semantic segmentation method using model uncertainty on indoor scene understanding using SUN RGB-D indoor scene dataset and achieves improvement of classification accuracy by this method.
Convolutional Networks with Bracket-Style Decoder for Semantic Scene Segmentation
TLDR
A deep convolutional network with bracket-style decoder, namely B-Net, is proposed to leverage the utilization of features learned at middle layers in the backbone networks (encoder) for constructing a final prediction map of densely enhanced semantic information.
Introspective Failure Prediction for Semantic Image Segmentation
TLDR
In a precision-recall analysis, the proposed method outperforms two state-of-the-art uncertainty estimation methods by 3.2% and 6.7% while requiring significantly less resources during inference.
Bayesian Uncertainty Quantification with Synthetic Data
TLDR
This paper conducts two sets of experiments to investigate the influence of distance, occlusion, clouds, rain, and puddles on the estimated uncertainty in the segmentation of road scenes and finds that the estimated aleatoric uncertainty from Bayesian deep models can be reduced with more training data.
KIL: Knowledge Interactiveness Learning for Joint Depth Estimation and Semantic Segmentation
TLDR
The proposed unified framework can mine and leverage the connections/complementations of these two tasks effectively and the network parameters can be jointly optimized for boosting the final predictions of depth estimation and semantic segmentation with the coarse-to-fine strategy.
Bayesian U-Net: Estimating Uncertainty in Semantic Segmentation of Earth Observation Images
TLDR
This work addresses uncertainty estimations in semantic segmentation with a Bayesian deep learning method, based on Monte Carlo Dropout, which allows for the performance of a sounder qualitative evaluation of the segmentation results and includes valuable information to improve the reference databases.
Evaluating Bayesian Deep Learning Methods for Semantic Segmentation
TLDR
This work modify DeepLab-v3+, one of the state-of-the-art deep neural networks, and create its Bayesian counterpart using MC dropout and Concrete dropout as inference techniques, and compares and test these two inference techniques on the well-known Cityscapes dataset using the suggested metrics.
S3Net: 3D LiDAR Sparse Semantic Segmentation Network
TLDR
S3Net is proposed, a novel convolutional neural network for LiDAR point cloud semantic segmentation that adopts an encoder-decoder backbone that consists of Sparse Intra-channel Attention Module (SIntraAM), and Sparse Inter-channel attention module (SInterAM) to emphasize the fine details of both within each feature map and among nearby feature maps.
Scalable Uncertainty for Computer Vision With Functional Variational Inference
TLDR
This work leverage the formulation of variational inference in function space, where Gaussian Processes are associated to both Bayesian CNN priors and variational family, and provides sufficient conditions for constructing regression loss functions whose probabilistic counterparts are compatible with aleatoric uncertainty quantification.
...
1
2
3
4
5
...

References

SHOWING 1-10 OF 52 REFERENCES
Modelling uncertainty in deep learning for camera relocalization
  • Alex Kendall, R. Cipolla
  • Computer Science
    2016 IEEE International Conference on Robotics and Automation (ICRA)
  • 2016
TLDR
A Bayesian convolutional neural network is used to regress the 6-DOF camera pose from a single RGB image and an estimate of the model's relocalization uncertainty is obtained to improve state of the art localization accuracy on a large scale outdoor dataset.
SegNet: A Deep Convolutional Encoder-Decoder Architecture for Robust Semantic Pixel-Wise Labelling
TLDR
The results show that SegNet achieves state-of-the-art performance even without use of additional cues such as depth, video frames or post-processing with CRF models.
Learning Deconvolution Network for Semantic Segmentation
TLDR
A novel semantic segmentation algorithm by learning a deep deconvolution network on top of the convolutional layers adopted from VGG 16-layer net, which demonstrates outstanding performance in PASCAL VOC 2012 dataset.
What Uncertainties Do We Need in Bayesian Deep Learning for Computer Vision?
TLDR
A Bayesian deep learning framework combining input-dependent aleatoric uncertainty together with epistemic uncertainty is presented, which makes the loss more robust to noisy data, also giving new state-of-the-art results on segmentation and depth regression benchmarks.
Fully Convolutional Networks for Semantic Segmentation
TLDR
It is shown that convolutional networks by themselves, trained end- to-end, pixels-to-pixels, improve on the previous best result in semantic segmentation.
SegNet: A Deep Convolutional Encoder-Decoder Architecture for Image Segmentation
TLDR
Quantitative assessments show that SegNet provides good performance with competitive inference time and most efficient inference memory-wise as compared to other architectures, including FCN and DeconvNet.
Combining Appearance and Structure from Motion Features for Road Scene Understanding
TLDR
A framework for pixel-wise object segmentation of road scenes that combines motion and appearance features that is designed to handle street-level imagery such as that on Google Street View and Microsoft Bing Maps is presented.
Semantic Image Segmentation with Deep Convolutional Nets and Fully Connected CRFs
TLDR
This work brings together methods from DCNNs and probabilistic graphical models for addressing the task of pixel-level classification by combining the responses at the final DCNN layer with a fully connected Conditional Random Field (CRF).
Multi-task Learning Using Uncertainty to Weigh Losses for Scene Geometry and Semantics
TLDR
A principled approach to multi-task deep learning is proposed which weighs multiple loss functions by considering the homoscedastic uncertainty of each task, allowing us to simultaneously learn various quantities with different units or scales in both classification and regression settings.
Conditional Random Fields as Recurrent Neural Networks
TLDR
A new form of convolutional neural network that combines the strengths of Convolutional Neural Networks (CNNs) and Conditional Random Fields (CRFs)-based probabilistic graphical modelling is introduced, and top results are obtained on the challenging Pascal VOC 2012 segmentation benchmark.
...
1
2
3
4
5
...