Image Inpainting Guided by Coherence Priors of Semantics and Textures

  title={Image Inpainting Guided by Coherence Priors of Semantics and Textures},
  author={Liang Liao and Jing Xiao and Zongge Wang and Chia-Wen Lin and Shin’ichi Satoh},
  journal={2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)},
  • Liang Liao, Jing Xiao, +2 authors S. Satoh
  • Published 15 December 2020
  • Computer Science
  • 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)
Existing inpainting methods have achieved promising performance in recovering defective images of specific scenes. However, filling holes involving multiple semantic categories remains challenging due to the obscure se-mantic boundaries and the mixture of different semantic textures. In this paper, we introduce coherence priors between the semantics and textures which make it possible to concentrate on completing separate textures in a semantic-wise manner. Specifically, we adopt a multi-scale… Expand

Figures and Tables from this paper

Bridging Global Context Interactions for High-Fidelity Image Completion
This paper proposes to treat image completion as a directionless sequence-to-sequence prediction task, and deploy a transformer to directly capture long-range dependence in the encoder, employing a restrictive CNN with small and non-overlapping RF for weighted token representation. Expand
SketchEdit: Mask-Free Local Image Manipulation with Partial Sketches
  • Yu Zeng, Zhe Lin, Vishal M. Patel
  • Computer Science
  • ArXiv
  • 2021
This work investigates a new paradigm of sketch-based image manipulation: maskfree local image manipulation, which only requires sketch inputs from users and utilizes the entire original image. Expand


Uncertainty-Aware Semantic Guidance and Estimation for Image Inpainting
A SEmantic GUidance and Estimation Network (SeGuE-Net) that iteratively evaluates the uncertainty of inpainted visual contents based on pixel-wise semantic inference and optimize structural priors and inpainted contents alternatively is proposed. Expand
Coherent Semantic Attention for Image Inpainting
This work investigates the human behavior in repairing pictures and proposes a fined deep generative model-based approach with a novel coherent semantic attention (CSA) layer, which can not only preserve contextual structure but also make more effective predictions of missing parts by modeling the semantic relevance between the holes features. Expand
High-Resolution Image Inpainting Using Multi-scale Neural Patch Synthesis
This work proposes a multi-scale neural patch synthesis approach based on joint optimization of image content and texture constraints, which not only preserves contextual structures but also produces high-frequency details by matching and adapting patches with the most similar mid-layer feature correlations of a deep classification network. Expand
Generative Image Inpainting with Contextual Attention
This work proposes a new deep generative model-based approach which can not only synthesize novel image structures but also explicitly utilize surrounding image features as references during network training to make better predictions. Expand
SPG-Net: Segmentation Prediction and Guidance Network for Image Inpainting
This paper proposes to introduce the semantic segmentation information, which disentangles the inter-class difference and intra-class variation for image inpainting, which leads to much clearer recovered boundary between semantically different regions and better texture within semantically consistent segments. Expand
Contextual-Based Image Inpainting: Infer, Match, and Translate
This work proposes a learning-based approach to generate visually coherent completion given a high-resolution image with missing components and shows that it generates results of better visual quality than previous state-of-the-art methods. Expand
Foreground-Aware Image Inpainting
  • Wei Xiong, Jiahui Yu, +4 authors Jiebo Luo
  • Computer Science
  • 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)
  • 2019
This work proposes a foreground-aware image inpainting system that explicitly disentangles structure inference and content completion, and shows that by such disentanglement, the contour completion model predicts reasonable contours of objects, and further substantially improves the performance of image inPainting. Expand
StructureFlow: Image Inpainting via Structure-Aware Appearance Flow
A two-stage model which splits the inpainting task into two parts: structure reconstruction and texture generation is proposed, which shows superior performance on multiple publicly available datasets. Expand
Image Inpainting With Learnable Bidirectional Attention Maps
This paper presents a learnable attention map module for learning feature re-normalization and mask-updating in an end-to-end manner, which is effective in adapting to irregular holes and propagation of convolution layers. Expand
Region filling and object removal by exemplar-based image inpainting
The simultaneous propagation of texture and structure information is achieved by a single, efficient algorithm that combines the advantages of two approaches: exemplar-based texture synthesis and block-based sampling process. Expand