Cross-MPI: Cross-scale Stereo for Image Super-Resolution using Multiplane Images

@article{Zhou2020CrossMPICS,
  title={Cross-MPI: Cross-scale Stereo for Image Super-Resolution using Multiplane Images},
  author={Yuemei Zhou and Gaochang Wu and Ying Fu and Kun Li and Yebin Liu},
  journal={2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)},
  year={2020},
  pages={14837-14846}
}
  • Yuemei ZhouGaochang Wu Yebin Liu
  • Published 30 November 2020
  • Computer Science
  • 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)
Various combinations of cameras enrich computational photography, among which reference-based super-resolution (RefSR) plays a critical role in multiscale imaging systems. However, existing RefSR approaches fail to accomplish high-fidelity super-resolution under a large resolution gap, e.g., 8× upscaling, due to the lower consideration of the underlying scene structure. In this paper, we aim to solve the RefSR problem in actual multiscale camera systems inspired by multiplane image (MPI… 
[PDF] Semantic Reader
9 Citations
Background Citations
5
Methods Citations
5

Figures and Tables from this paper

9 Citations

LocalTrans: A Multiscale Local Transformer Network for Cross-Resolution Homography Estimation

Experiments show that the proposed network outperforms existing state-of-the-art feature-based and deep-learning-based homography estimation methods, and is able to accurately align images under 10× resolution gap.

An end-to-end deep convolutional neural network for image restoration of sparse aperture imaging system in geostationary orbit

The development of large-aperture telescopes employing monolithic mirrors has been greatly limited by technical constraints and the difficulty of processing and manufacturing. The sparse aperture…

MC-ViT: Multi-path cross-scale vision transformer for thymoma histopathology whole slide image typing

A deep learning-based thymoma typing method for hematoxylin & eosin (H&E)-stained whole slide images (WSIs), which provides useful histopathology information from patients to assist doctors for better diagnosingThymoma or TC and may assist doctors in improving diagnosis efficiency and accuracy.

Flexible Hybrid Lenses Light Field Super-Resolution using Layered Refinement

A novel learning-based framework with Layered Refinement to super-resolve the hybrid lenses LF images and outperforms the SOTA methods in kinds of scenes from simulated and real-world datasets with various disparity ranges.

Guided Hyperspectral Image Denoising with Realistic Data

The extensive experimental results show that a network learned with only synthetic data generated by the noise model performs as well as it is learned with paired real data, and the guided HSI denoising network outperforms state-of-the-art methods under both quantitative metrics and visual quality.

Latent Multi-Relation Reasoning for GAN-Prior based Image Super-Resolution

LAREN is a LAtent multi- Relation rEasoNing technique that achieves superb large-factor SR through graph-based multi-relation reasoning in latent space and outperforms the state-of-the-art consistently across multiple benchmarks.

Geo-NI: Geometry-aware Neural Interpolation for Light Field Rendering

By combining the superiorities of NI and DIBR, the proposed Geo-NI is able to render views with large disparity with the help of scene geometry while also reconstruct non-Lambertian effect when depth is prone to be ambiguous.

A survey on multimodal-guided visual content synthesis

Attention Mechanism-Based Light-Field View Synthesis

This article presents a novel deep learning-based light-field view synthesis method from a sparse set of input views that utilizes convolutional block attention modules to enhance the built-in depth image-based rendering.

34 References

Cross-Scale Reference-Based Light Field Super-Resolution

To solve the nontrivial warping problem that induced by the significant resolution gaps between the cross-scale inputs, multiple disparity maps from the reference image to all the LR light field images, followed by a blending strategy to fuse for a refined disparity map; finally, a high-quality super-resolved light field can be obtained.

CrossNet: An End-to-end Reference-based Super Resolution Network using Cross-scale Warping

Using cross-scale warping, the CrossNet network is able to perform spatial alignment at pixel-level in an end-to-end fashion, which improves the existing schemes both in precision and efficiency.

Multiscale gigapixel video: A cross resolution image matching and warping approach

Experimental results show that the proposed multi-scale camera array and cross resolution video warping scheme is capable of generating seamless gigapixel video without the need of camera calibration and large overlapping area constraints between the local-view cameras.

Image Super-Resolution by Neural Texture Transfer

An end-to-end deep model which enriches HR details by adaptively transferring the texture from Ref images according to their textural similarity is designed, which facilitates multi-scale neural transfer that allows the model to benefit more from those semantically related Ref patches, and gracefully degrade to SISR performance on the least relevant Ref inputs.

Learning Parallax Attention for Stereo Image Super-Resolution

A parallax-attention mechanism with a global receptive field along the epipolar line to handle different stereo images with large disparity variations is introduced and a new and the largest dataset for stereo image SR is proposed.

Pushing the Boundaries of View Extrapolation With Multiplane Images

This paper presents a theoretical analysis showing how the range of views that can be rendered from an MPI increases linearly with the MPI disparity sampling frequency, as well as a novel MPI prediction procedure that theoretically enables view extrapolations of up to 4 times the lateral viewpoint movement allowed by prior work.

Structure-Preserving Super Resolution With Gradient Guidance

A structure-preserving super resolution method which exploits gradient maps of images to guide the recovery in two aspects and proposes a gradient loss which imposes a second-order restriction on the super-resolved images.

Learning Cross-scale Correspondence and Patch-based Synthesis for Reference-based Super-Resolution

Experiments on MPI Sintel Dataset and Light-Field video dataset demonstrate the learned correspondence features outperform existing features, and the proposed RefSR-Net substantially outperforms conventional single image SR and exemplar-based SR approaches.

EnhanceNet: Single Image Super-Resolution Through Automated Texture Synthesis

This work proposes a novel application of automated texture synthesis in combination with a perceptual loss focusing on creating realistic textures rather than optimizing for a pixelaccurate reproduction of ground truth images during training to achieve a significant boost in image quality at high magnification ratios.

Stereo Magnification: Learning View Synthesis using Multiplane Images

This paper explores an intriguing scenario for view synthesis: extrapolating views from imagery captured by narrow-baseline stereo cameras, including VR cameras and now-widespread dual-lens camera phones, and proposes a learning framework that leverages a new layered representation that is called multiplane images (MPIs).