Corpus ID: 227228603

RGBD-Net: Predicting color and depth images for novel views synthesis

@article{Nguyen2020RGBDNetPC,
  title={RGBD-Net: Predicting color and depth images for novel views synthesis},
  author={Phong Nguyen and Animesh Karnewar and Lam Huynh and Esa Rahtu and Jiri Matas and J. Heikkila},
  journal={ArXiv},
  year={2020},
  volume={abs/2011.14398}
}
We address the problem of novel view synthesis from an unstructured set of reference images. A new method called RGBD-Net is proposed to predict the depth map and the color images at the target pose in a multi-scale manner. The reference views are warped to the target pose to obtain multi-scale plane sweep volumes, which are then passed to our first module, a hierarchical depth regression network which predicts the depth map of the novel view. Second, a depth-aware generator network refines the… Expand
FaDIV-Syn: Fast Depth-Independent View Synthesis
TLDR
The multi-view approach addresses the problem that view synthesis methods are often limited by their depth estimation stage, where incorrect depth predictions can lead to large projection errors and efficiently warp multiple input images into the target frame for a range of assumed depth planes. Expand

References

SHOWING 1-10 OF 70 REFERENCES
MVSNet: Depth Inference for Unstructured Multi-view Stereo
TLDR
This work presents an end-to-end deep learning architecture for depth map inference from multi-view images that flexibly adapts arbitrary N-view inputs using a variance-based cost metric that maps multiple features into one cost feature. Expand
BlendedMVS: A Large-Scale Dataset for Generalized Multi-View Stereo Networks
  • Yao Yao, Zixin Luo, +5 authors Long Quan
  • Computer Science
  • 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)
  • 2020
TLDR
This paper introduces BlendedMVS, a novel large-scale dataset to provide sufficient training ground truth for learning-based MVS and endows the trained model with significantly better generalization ability compared with other MVS datasets. Expand
Point-Based Multi-View Stereo Network
TLDR
This work introduces Point-MVSNet, a novel point-based deep framework for multi-view stereo (MVS), which directly processes the target scene as point clouds and allows higher accuracy, more computational efficiency and more flexibility than cost-volume-based counterparts. Expand
Learning-based view synthesis for light field cameras
TLDR
This paper proposes a novel learning-based approach to synthesize new views from a sparse set of input views that could potentially decrease the required angular resolution of consumer light field cameras, which allows their spatial resolution to increase. Expand
Deep Stereo Using Adaptive Thin Volume Representation With Uncertainty Awareness
  • Shuo Cheng, Zexiang Xu, +4 authors Hao Su
  • Computer Science
  • 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)
  • 2020
TLDR
The proposed ATV consists of only a small number of planes with low memory and computation costs; yet, it efficiently partitions local depth ranges within learned small uncertainty intervals, which enables reconstruction with high completeness and accuracy in a coarse-to-fine fashion. Expand
Deep blending for free-viewpoint image-based rendering
TLDR
This work presents a new deep learning approach to blending for IBR, in which held-out real image data is used to learn blending weights to combine input photo contributions, and designs the network architecture and the training loss to provide high quality novel view synthesis, while reducing temporal flickering artifacts. Expand
Stereo Magnification: Learning View Synthesis using Multiplane Images
TLDR
This paper explores an intriguing scenario for view synthesis: extrapolating views from imagery captured by narrow-baseline stereo cameras, including VR cameras and now-widespread dual-lens camera phones, and proposes a learning framework that leverages a new layered representation that is called multiplane images (MPIs). Expand
Free View Synthesis
TLDR
This work presents a method for novel view synthesis from input images that are freely distributed around a scene that can synthesize images for free camera movement through the scene, and works for general scenes with unconstrained geometric layouts. Expand
Depth synthesis and local warps for plausible image-based navigation
TLDR
This work introduces a new IBR algorithm that is robust to missing or unreliable geometry, providing plausible novel views even in regions quite far from the input camera positions, and demonstrates novel view synthesis in real time for multiple challenging scenes with significant depth complexity. Expand
DeepMVS: Learning Multi-view Stereopsis
TLDR
The results show that DeepMVS compares favorably against state-of-the-art conventional MVS algorithms and other ConvNet based methods, particularly for near-textureless regions and thin structures. Expand
...
1
2
3
4
5
...