Depth Image Based View Synthesis: New Insights and Perspectives on Hole Generation and Filling

@article{Zhu2016DepthIB,
  title={Depth Image Based View Synthesis: New Insights and Perspectives on Hole Generation and Filling},
  author={Ce Zhu and Shuai Li},
  journal={IEEE Transactions on Broadcasting},
  year={2016},
  volume={62},
  pages={82-93}
}
  • Ce Zhu, Shuai Li
  • Published 1 March 2016
  • Computer Science
  • IEEE Transactions on Broadcasting
View synthesis with depth-image-based rendering (DIBR) has attracted great interest in that it can provide a virtual image at any arbitrary viewpoint in 3-D video and free-viewpoint TV. An inherent problem in the DIBR view synthesis is occurrence of holes in a synthesized image, which is also known as disocclusion problem. The disoccluded regions need to be handled properly in order to generate a synthesized view of good quality. This paper provides a fundamental examination of hole generation… 
Hole Filling With Multiple Reference Views in DIBR View Synthesis
TLDR
The view interpolation with multiple reference views is examined, demonstrating that the problem of emerging holes in a target virtual view can be greatly alleviated by making good use of other neighboring complementary views in addition to its two (commonly used) most neighboring primary views.
Hole Filling for View Synthesis Using Depth Guided Global Optimization
TLDR
A hole filling method with depth-guided global optimization is proposed for view synthesis and has better performance compared with other methods in terms of visual quality, trusted textures, and temporal consistency in the synthesized video.
Color Correction and Depth-Based Hierarchical Hole Filling in Free Viewpoint Generation
TLDR
Experimental results show that the proposed model can produce high-quality virtual views with comfortable visual perceptions and Comparisons with other state-of-the-art models demonstrate that the method can achieve good performance in both visual quality and quantitative evaluation.
Artifact Handling Based on Depth Image for View Synthesis
TLDR
An artifact handling method based on depth image is proposed, which has better performance compared with previous methods in subjective and objective evaluation.
Fast MRF-Based Hole Filling for View Synthesis
TLDR
In this letter, a fast MRF-based hole filling method is proposed for view synthesis, which is formulated as an energy minimization problem and is solved with loopy belief propagation (LBP).
Foreground Removal Approach for Hole Filling in 3D Video and FVV Synthesis
TLDR
A foreground removal approach for hole filling is proposed, in which the foreground objects are removed from both the 2D video and its corresponding depth map, and then a BG video is generated before the 3D warping and used to eliminate the holes in the synthesized video.
A novel inpainting framework for virtual view synthesis
TLDR
A new inpainting framework that innovatively exploits depth and textural self-similarity characteristics to construct subjectively enhanced virtual viewpoints is presented, showing superior performance both perceptually and numerically compared to existing techniques, especially in terms of lower visual artefacts.
Quality Index for View Synthesis by Measuring Instance Degradation and Global Appearance
TLDR
A quality index for view synthesis is presented by simultaneously measuring local Instance DEgradation and global Appearance (IDEA) and it is proposed to measure the global appearance of synthesized images based on the superpixel representation.
A Layered Approach for Quality Assessment of DIBR-Synthesized Images
TLDR
A 3D quality assessment metric using the depth of the scene divides the image into different layers where each layer represents the objects at a different distance from the camera, and their scores are pooled together to obtain a single quality score that represents the quality of the synthesized image.
Disocclusion-type aware hole filling method for view synthesis
TLDR
Experimental results indicate that the proposed disocclusion-type aware hole filling method outperforms the other methods in the objective and subjective evaluations.
...
...

References

SHOWING 1-10 OF 33 REFERENCES
A New Perspective on Hole Generation and Filling in DIBR Based View Synthesis
  • Ce Zhu, Shuai Li
  • Computer Science
    2013 Ninth International Conference on Intelligent Information Hiding and Multimedia Signal Processing
  • 2013
TLDR
A novel hole-filling method is proposed by fully utilizing the occlusion information that can effectively locate the relevant background pixels including those occluded around the hole and make use of integrated background information in the hole filling.
View Synthesis Based on Background Update with Gaussian Mixture Model
TLDR
An approach is proposed to fill the disocclustion filling after 3-D warping in view synthesis system by using the real background information covered in the original view, which is based on the observation that the covered information in the current frame may be visible in the next frames of the same view.
Boundary Artifact Reduction in View Synthesis of 3D Video: From Perspective of Texture-Depth Alignment
TLDR
A novel solution of suppression of misalignment and alignment enforcement between texture and depth to reduce background noises and foreground erosion, respectively, among different types of boundary artifacts is proposed.
Improved novel view synthesis from depth image with large baseline
TLDR
A new algorithm is developed for recovering the large disocclusion regions in depth image based rendering (DIBR) systems on 3DTV and a depth-guided exemplar-based image inpainting that combines the structural strengths of the color gradient to preserve the image structure in the restored regions is proposed.
Depth Map Driven Hole Filling Algorithm Exploiting Temporal Correlation Information
TLDR
The temporal correlation of texture and depth information is exploited to generate a background reference image that is then used to fill the holes associated with the dynamic parts of the scene, whereas for static parts the traditional inpainting method is used.
Hole filling method using depth based in-painting for view synthesis in free viewpoint television and 3-D video
TLDR
Experimental results show that the proposed hole filling method provides improved rendering quality both objectively and subjectively.
Depth-aided image inpainting for novel view synthesis
TLDR
This paper proposes an inpainting approach to retrieve missing pixels in the texture and structure propagation process to take into account the depth information by distinguishing foreground and background parts of the scene.
Free-viewpoint depth image based rendering
A new multidirectional extrapolation hole-filling method for Depth-Image-Based Rendering
TLDR
A novel multidirectional extrapolation hole-filling method is proposed to enhance visual quality for large holes with complex texture background and demonstrated that the proposed method could provide better visual quality compared with conventional methods for virtual views synthesis with high-quality depth map.
Adaptive edge-oriented depth image smoothing approach for depth image based rendering
  • Pei-Jun Lee, Effendi
  • Computer Science
    2010 IEEE International Symposium on Broadband Multimedia Systems and Broadcasting (BMSB)
  • 2010
TLDR
The proposed adaptive edge-oriented smoothing process adopts an edge dependent depth filter to predict the holes region and then the horizontal smoothing filter is first employed on the predicted holes regions to preserve the depth quality.
...
...