Probabilistic Warp Consistency for Weakly-Supervised Semantic Correspondences

@article{Truong2022ProbabilisticWC,
  title={Probabilistic Warp Consistency for Weakly-Supervised Semantic Correspondences},
  author={Prune Truong and Martin Danelljan and Fisher Yu and Luc Van Gool},
  journal={ArXiv},
  year={2022},
  volume={abs/2203.04279}
}
We propose Probabilistic Warp Consistency, a weaklysupervised learning objective for semantic matching. Our approach directly supervises the dense matching scores predicted by the network, encoded as a conditional probability distribution. We first construct an image triplet by applying a known warp to one of the images in a pair depicting different instances of the same object class. Our probabilistic learning objectives are then derived using the constraints arising from the resulting image… 

References

SHOWING 1-10 OF 51 REFERENCES
PatchMatch-Based Neighborhood Consensus for Semantic Correspondence
TLDR
This work has devised an approach to jointly train the scoring function and the feature extraction modules by embedding them into a proxy model which is end-to-end differentiable and significantly outperforms the state-of-the-art on both datasets while also being faster and using less memory.
Probabilistic Model Distillation for Semantic Correspondence
TLDR
A novel Probabilistic Model Distillation approach which transfers knowledge learned by a probabilistic teacher model on synthetic data to a static student model with the use of unlabeled real image pairs and is extended to incorporate stronger supervision for better accuracy.
End-to-End Weakly-Supervised Semantic Alignment
TLDR
A convolutional neural network architecture for semantic alignment that is trainable in an end-to-end manner from weak image-level supervision in the form of matching image pairs that computes the quality of the alignment based on only geometrically consistent correspondences thereby reducing the effect of background clutter.
SFNet: Learning Object-Aware Semantic Correspondence
TLDR
A new CNN architecture is proposed, dubbed SFNet, which leverages a new and differentiable version of the argmax function for end-to-end training, with a loss that combines mask and flow consistency with smoothness terms.
Dynamic Context Correspondence Network for Semantic Alignment
TLDR
This paper proposes a context-aware semantic representation that incorporates spatial layout for robust matching against local ambiguities and develops a novel dynamic fusion strategy based on attention mechanism to weave the advantages of both local and context features by integrating semantic cues from multiple scales.
Show, Match and Segment: Joint Weakly Supervised Learning of Semantic Matching and Object Co-Segmentation
TLDR
The model is end-to-end trainable and does not require supervision from manually annotated correspondences and object masks, and performs favorably against the state-of-the-art methods on both semantic matching and object co-segmentation tasks.
Learning Dense Correspondence via 3D-Guided Cycle Consistency
TLDR
It is demonstrated that the end-to-end trained ConvNet supervised by cycle-consistency outperforms state-of-the-art pairwise matching methods in correspondence-related tasks.
PDC-Net+: Enhanced Probabilistic Dense Correspondence Network
TLDR
The Enhanced Probabilistic Dense Correspondence Network, PDC-Net+, is proposed, capable of estimating accurate dense correspondences along with a reliable confidence map, and a flexible probabilistic approach that jointly learns the flow prediction and its uncertainty is developed.
Universal Correspondence Network
TLDR
A convolutional spatial transformer to mimic patch normalization in traditional features like SIFT is proposed, which is shown to dramatically boost accuracy for semantic correspondences across intra-class shape variations.
Attentive Semantic Alignment with Offset-Aware Correlation Kernels
TLDR
This paper introduces an attentive semantic alignment method that focuses on reliable correlations, filtering out distractors, and proposes an offset-aware correlation kernel that learns to capture translation-invariant local transformations in computing correlation values over spatial locations.
...
...