• Corpus ID: 53767318

SpherePHD: Applying CNNs on a Spherical PolyHeDron Representation of 360 degree Images

@article{Lee2018SpherePHDAC,
  title={SpherePHD: Applying CNNs on a Spherical PolyHeDron Representation of 360 degree Images},
  author={Yeonkun Lee and Jaeseok Jeong and Jong Seob Yun and Cho Won June and Kuk-jin Yoon},
  journal={ArXiv},
  year={2018},
  volume={abs/1811.08196}
}
Omni-directional cameras have many advantages overconventional cameras in that they have a much wider field-of-view (FOV). Accordingly, several approaches have beenproposed recently to apply convolutional neural networks(CNNs) to omni-directional images for various visual tasks.However, most of them use image representations defined inthe Euclidean space after transforming the omni-directionalviews originally formed in the non-Euclidean space. Thistransformation leads to shape distortion due to… 
Reprojection R-CNN: A Fast and Accurate Object Detector for 360° Images
TLDR
This paper adapts the terminologies of the traditional object detection task to the omnidirectional scenarios, and proposes a novel two-stage object detector, Reprojection R-CNN, which outperforms the previous state-of-the-art methods on the mAP metric.
Orientation-Aware Semantic Segmentation on Icosahedron Spheres
TLDR
This work proposes an orientation-aware CNN framework for the icosahedron mesh, which allows for fast network operations, as the design simplifies to standard network operations of classical CNNs, but under consideration of north-aligned kernel convolutions for features on the sphere.
FisheyeDet: A Self-Study and Contour-Based Object Detector in Fisheye Images
TLDR
The No-prior Fisheye Representation Method and the Distortion Shape Matching strategy is proposed, which invokes the irregular quadrilateral bounding boxes based on the contour of distorted objects as the core of the proposed object detector.
OmniFlowNet: a Perspective Neural Network Adaptation for Optical Flow Estimation in Omnidirectional Images
TLDR
The proposed network, OmniFlowNet, is built on a CNN specialized in perspective images that is adapted to be consistent with the equirectangular projection and shows better performance than its original network without extra training.
Spherical Image Generation from a Single Image by Considering Scene Symmetry
TLDR
A method for generating a spherical image from a single NFOV image and controlling the degree of freedom of the generated regions using the scene symmetry, which is a basic property of the global structure of spherical images, such as rotational symmetry, plane symmetry, and asymmetry.
MatryODShka: Real-time 6DoF Video View Synthesis using Multi-Sphere Images
TLDR
This work introduces a method to convert stereo 360° (omnidirectional stereo) imagery into a layered, multi-sphere image representation for six degree-of-freedom (6DoF) rendering, which can be rendered with correct 6DoF disparity and motion parallax in VR.
Spherical Convolutionnal Neural Networks: Empirical Analysis of SCNNs
TLDR
The DeepSphere model approximates the sphere with a graph and performs graph convolutions, and is evaluated against other spherical CNNs on different tasks to prove the flexibility of DeepSphere with a non-hierarchical and irregular sampling of the sphere.
FDDB-360: Face Detection in 360-Degree Fisheye Images
TLDR
It is shown how a face detector trained on regular images can be re-trained for this purpose, and a 360° fisheye-like version of the popular FDDB face detection dataset is provided, which is called FDDB-360.
HoHoNet: 360 Indoor Holistic Understanding with Latent Horizontal Features
TLDR
HoNet is a versatile and efficient framework for holistic understanding of an indoor 360-degree panorama using a Latent Horizontal Feature (LHFeat), and a novel horizon-to-dense module, which relaxes the per-column output shape constraint, allowing per-pixel dense prediction from LHFeat.
A Temporally Consistent Image-based Sun Tracking Algorithm for Solar Energy Forecasting Applications
TLDR
This study introduces an image-based Sun tracking algorithm to localise the Sun in the image when it is visible and interpolate its daily trajectory from past observations, and shows that the proposed method provides robust smooth Sun trajectories with a mean absolute error below 1% of the image size.
...
1
2
...

References

SHOWING 1-10 OF 25 REFERENCES
SphereNet: Learning Spherical Representations for Detection and Classification in Omnidirectional Images
TLDR
This work presents SphereNet, a novel deep learning framework which encodes invariance against such distortions explicitly into convolutional neural networks, and enables the transfer of existing perspective convolutionAL neural network models to the omnidirectional case.
Spherical CNNs
TLDR
A definition for the spherical cross-correlation is proposed that is both expressive and rotation-equivariant and satisfies a generalized Fourier theorem, which allows us to compute it efficiently using a generalized (non-commutative) Fast Fourier Transform (FFT) algorithm.
Learning Spherical Convolution for Fast Features from 360° Imagery
TLDR
This work proposes to learn a spherical convolutional network that translates a planar CNN to process 360{\deg} imagery directly in its equirectangular projection, and yields the most accurate results while saving orders of magnitude in computation versus the existing exact reprojection solution.
Cube Padding for Weakly-Supervised Saliency Prediction in 360° Videos
TLDR
A spatial-temporal network which is (1) weakly-supervised trained and (2) tailor-made for 360° viewing sphere, and outperforms baseline methods in both speed and quality.
Distortion-Aware Convolutional Filters for Dense Prediction in Panoramic Images
TLDR
This work proposes a learning approach for panoramic depth map estimation from a single image, thanks to a specifically developed distortion-aware deformable convolution filter, which can be trained by means of conventional perspective images, then used to regress depth forPanoramic images, thus bypassing the effort needed to create annotated pan oramic training dataset.
SalNet360: Saliency Maps for omni-directional images with CNN
TLDR
This paper presents an architectural extension to any Convolutional Neural Network to fine-tune traditional 2D saliency prediction to Omnidirectional Images (ODIs) in an end-to-end manner and shows that each step in the proposed pipeline works towards making the generated saliency map more accurate with respect to ground truth data.
Object Detection in Equirectangular Panorama
TLDR
A high-resolution equirectangular panorama dataset for object detection and a multi-projection variant of the YOLO detector, which outperforms the other state-of-the-art detector, Faster R-CNN, and achieves the best accuracy with low-resolution input.
The SYNTHIA Dataset: A Large Collection of Synthetic Images for Semantic Segmentation of Urban Scenes
TLDR
This paper generates a synthetic collection of diverse urban images, named SYNTHIA, with automatically generated class annotations, and conducts experiments with DCNNs that show how the inclusion of SYnTHIA in the training stage significantly improves performance on the semantic segmentation task.
Semantic-Driven Generation of Hyperlapse from 360 Degree Video
TLDR
A system for converting a fully panoramic video into a normal field-of-view (NFOV) hyperlapse for an optimal viewing experience that exploits visual saliency and semantics to non-uniformly sample in space and time for generating hyperlapses.
Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks
TLDR
This work introduces a Region Proposal Network (RPN) that shares full-image convolutional features with the detection network, thus enabling nearly cost-free region proposals and further merge RPN and Fast R-CNN into a single network by sharing their convolutionAL features.
...
1
2
3
...