Tangent Images for Mitigating Spherical Distortion

@article{Eder2020TangentIF,
  title={Tangent Images for Mitigating Spherical Distortion},
  author={Marc Eder and Mykhailo Shvets and John Lim and Jan-Michael Frahm},
  journal={2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)},
  year={2020},
  pages={12423-12431}
}
In this work, we propose "tangent images," a spherical image representation that facilitates transferable and scalable 360 degree computer vision. Inspired by techniques in cartography and computer graphics, we render a spherical image to a set of distortion-mitigated, locally-planar image grids tangent to a subdivided icosahedron. By varying the resolution of these grids independently of the subdivision level, we can effectively represent high resolution spherical images while still benefiting… Expand
A comparative study of semantic segmentation using omnidirectional images
The semantic segmentation of omnidirectional urban driving images is a research topic that has increasingly attracted the attention of researchers. This paper presents a thorough comparative study ofExpand
Quality Assessment of Super-Resolved Omnidirectional Image Quality Using Tangential Views
TLDR
An objective, full-reference quality assessment framework which studies quality measurement for ODIs generated by GAN-based and CNN-based SISR methods is proposed and reveals that most objective metric show high performance over CNN based S ISR, while subjective tests favors Gan-based architectures. Expand
360MonoDepth: High-Resolution 360{\deg} Monocular Depth Estimation
TLDR
This work proposes a flexible framework for monocular depth estimation from high-resolution 360° images using tangent images, and recombine the individual depth estimates using deformable multi-scale alignment followed by gradient-domain blending to improve the consistency of disparity estimates. Expand
360MonoDepth: High-Resolution 360° Monocular Depth Estimation
360° cameras can capture complete environments in a single shot, which makes 360° imagery alluring in many computer vision tasks. However, monocular depth estimation remains a challenge for 360°Expand
ASOD60K: An Audio-Induced Salient Object Detection Dataset for Panoramic Videos
  • Yi Zhang
  • Computer Science
  • 2021
TLDR
PV-SOD, a new task that aims to segment salient objects from panoramic videos, and collects the first large-scale dataset, named ASOD60K, which contains 4K-resolution video frames annotated with a six-level hierarchy, thus distinguishing itself with richness, diversity and quality. Expand
ASOD60K: Audio-Induced Salient Object Detection in Panoramic Videos
TLDR
PV-SOD, a new task that aims to segment salient objects from panoramic videos, is proposed and the first large-scale dataset, named ASOD60K, is collected, which contains 4K-resolution video frames annotated with a six-level hierarchy, thus distinguishing itself with richness, diversity and quality. Expand
Applying VertexShuffle Toward 360-Degree Video Super-Resolution on Focused-Icosahedral-Mesh
  • Na Li, Yao Liu
  • Computer Science
  • ArXiv
  • 2021
TLDR
This work proposes a spherical super-resolution model that directly operates on a mesh representation of spherical pixels of 360-degree data and proposes a novel VertexShuffle operation that can significantly improve both the performance and the efficiency compared to the original MeshConv Transpose operation. Expand
Detecting Low-Rank Regions in Omnidirectional Images *
TLDR
A method to generate a low-rank probability map on an omnidirectional image and use it to build a training set in a self-supervised manner to train deep models to predict low- rank likelihood maps for omnid Directional images is proposed. Expand
Distortion-Aware Monocular Depth Estimation for Omnidirectional Images
TLDR
A Distortion-Aware Monocular Omnidirectional (DAMO) network to estimate dense depth maps from indoor panoramas is proposed and achieves the state-of-the-art performance on the 360D dataset with high efficiency. Expand
...
1
2
...

References

SHOWING 1-10 OF 32 REFERENCES
Orientation-Aware Semantic Segmentation on Icosahedron Spheres
TLDR
This work proposes an orientation-aware CNN framework for the icosahedron mesh, which allows for fast network operations, as the design simplifies to standard network operations of classical CNNs, but under consideration of north-aligned kernel convolutions for features on the sphere. Expand
Gauge Equivariant Convolutional Networks and the Icosahedral CNN
TLDR
Gauge equivariant convolution using a single conv2d call is demonstrated, making it a highly scalable and practical alternative to Spherical CNNs and demonstrating substantial improvements over previous methods on the task of segmenting omnidirectional images and global climate patterns. Expand
SpherePHD: Applying CNNs on a Spherical PolyHeDron Representation of 360° Images
TLDR
A novel method utilizes a spherical polyhedron to rep-resent omni-directional views and minimizes thevariance of the spatial resolving power on the sphere sur-face, and includes new convolution and pooling methods for the proposed representation. Expand
Spherical CNNs on Unstructured Grids
TLDR
Overall, a novel CNN approach on unstructured grids using parameterized differential operators for spherical signals is presented and it is shown that the unique kernel parameterization allows the model to achieve the same or higher accuracy with significantly fewer network parameters. Expand
Distortion-Aware Convolutional Filters for Dense Prediction in Panoramic Images
TLDR
This work proposes a learning approach for panoramic depth map estimation from a single image, thanks to a specifically developed distortion-aware deformable convolution filter, which can be trained by means of conventional perspective images, then used to regress depth forPanoramic images, thus bypassing the effort needed to create annotated pan oramic training dataset. Expand
Spherical CNNs
TLDR
A definition for the spherical cross-correlation is proposed that is both expressive and rotation-equivariant and satisfies a generalized Fourier theorem, which allows us to compute it efficiently using a generalized (non-commutative) Fast Fourier Transform (FFT) algorithm. Expand
Joint 2D-3D-Semantic Data for Indoor Scene Understanding
TLDR
A dataset of large-scale indoor spaces that provides a variety of mutually registered modalities from 2D, 2.5D and 3D domains, with instance-level semantic and geometric annotations, enables development of joint and cross-modal learning models and potentially unsupervised approaches utilizing the regularities present in large- scale indoor spaces. Expand
Convolutions on Spherical Images
TLDR
A representation for deep spherical image inference based on the icosahedral Snyder equal-area projection, a projection onto a geodesic grid, and show that it vastly exceeds the state-of-the-art for convolution on spherical images, improving semantic segmentation results by 12.6%. Expand
Kernel Transformer Networks for Compact Spherical Convolution
  • Yu-Chuan Su, K. Grauman
  • Computer Science
  • 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)
  • 2019
TLDR
The Kernel Transformer Network (KTN) is presented to efficiently transfer convolution kernels from perspective images to the equirectangular projection of 360° images and successfully preserves the source CNN’s accuracy, while offering transferability, scalability to typical image resolutions, and, in many cases, a substantially lower memory footprint. Expand
Planet texture maps, 2019
  • http:// planetpixelemporium.com/earth8081.html. Accessed 2019-04-16
  • 2019
...
1
2
3
4
...