Joint Video Object Discovery and Segmentation by Coupled Dynamic Markov Networks

  title={Joint Video Object Discovery and Segmentation by Coupled Dynamic Markov Networks},
  author={Zi-yi Liu and Le Wang and Gang Hua and Qilin Zhang and Zhenxing Niu and Ying Wu and Nanning Zheng},
  journal={IEEE Transactions on Image Processing},
It is a challenging task to extract segmentation mask of a target from a single noisy video, which involves object discovery coupled with segmentation. To solve this challenge, we present a method to jointly discover and segment an object from a noisy video, where the target disappears intermittently throughout the video. Previous methods either only fulfill video object discovery, or video object segmentation presuming the existence of the object in each frame. We argue that jointly conducting… 

Joint Multi-object Detection and Segmentation from an Untrimmed Video

A novel method for jointly detecting and segmenting multiple objects from an untrimmed video where the target objects do not always appear per frame and outperforms most of the state-of-the-art approaches.

Object Cosegmentation in Noisy Videos With Multilevel Hypergraph

A multilevel hypergraph-based full Video object CoSegmentation (VCS) method, which incorporates high-level semantics and low-level appearance/motion/saliency to construct the hyperedge among multiple spatially and temporally adjacent regions is proposed.

Deep Pixel-Level Matching via Attention for Video Co-Segmentation

A framework based on deep neural networks and equipped with a new attention module, which is designed for pixel-level matching to segment the object across video frames in this paper, achieves higher accuracy than those of other video segmentation methods that perform well in common information extraction.

Action Co-localization in an Untrimmed Video by Graph Neural Networks

The Graph-based Temporal Action Co-Localization (G-TACL) method, which aggregates contextual features from multiple action proposals to assist temporal localization, is proposed, which combines Graph Neural Networks with nodes initialized by the action proposal representations.

Action Coherence Network for Weakly Supervised Temporal Action Localization

This work presents Action Coherence Network (ACN) for W-TAL, which features a new coherence loss that better supervises action boundary learning and facilitate proposal regression and a purpose-built fusion module is proposed for localization inference based on features extracted by two streams of convolutional neural network.

Object Affordances Graph Network for Action Recognition

The Object Affordances Graph (OAG), which cast human-object interaction cues into video representations via an iterative refinement procedure, is proposed, and the proposed OAGN, which outperforms existing state-of-the-art affordance-less action recognition methods.

Calibrank: Effective Lidar-Camera Extrinsic Calibration By Multi-Modal Learning To Rank

The learning-to-rank pipeline is introduced for ranking the top n relevant poses in the gallery set, which is then fused in to the final prediction, and an exponential mapping from parametric space to the relevance space is proposed.

EDC3: Ensemble of Deep-Classifiers using Class-specific Copula functions to Improve Semantic Image Segmentation

A class-wise Copula-based ensembling method is newly proposed for solving the multi-class segmentation problem and it is observed that the performance has improved more for semantic image segmentation using the proposed class-specific Copula function than the traditionally used single Copulafunction for the problem.

Machine learning in geoscience: using deep learning to solve the TGS Salt Identification challenge

This paper successfully applied an U-net based image semantic segmentation to identify salt bodies using only seismic images from the TGS Salt Identification Challenge.

GasHolesJ- a software tool for measurement of gas holes in cheese

The results show that six algorithms for global auto thresholding are effective enough for detection gas holes in Emmental cheese.



Video Object Discovery and Co-Segmentation with Extremely Weak Supervision

The proposed spatio-temporal energy minimization formulation for simultaneous video object discovery and co-segmentation across multiple videos containing irrelevant frames compares favorably with the state-of-the-art in all of these experiments.

Video Object Co-segmentation by Regulated Maximum Weight Cliques

A novel approach for object co-segmentation in arbitrary videos by sampling, tracking and matching object proposals via a Regulated Maximum Weight Clique (RMWC) extraction scheme that can handle multiple objects, temporary occlusions, and objects going in and out of view.

Extracting Primary Objects by Video Co-Segmentation

This work proposes an algorithm that learns the model of the primary object by representing the frames/videos as a graphical model, built across a set of videos based on an object proposal algorithm.

Video Object Segmentation through Spatially Accurate and Temporally Dense Extraction of Primary Object Regions

The extracted primary object regions are then used to build object models for optimized video segmentation and outperforms both unsupervised and supervised state-of-the-art methods.

Multi-class Video Co-segmentation with a Generative Multi-video Model

This work formulates a non-parametric Bayesian model across videos sequences that is based on a new videos segmentation prior as well as a global appearance model that links segments of the same class in multi-class video co-segmentation.

Video Object Co-Segmentation from Noisy Videos by a Multi-Level Hypergraph Model

A new multilevel hypergraph based method, termed the full Video object Co-Segmentation method (VCS), is proposed, which incorporates both a high-level semantics object model and a low-level appearance/motion/saliency object model to construct the hyperedge among multiple spatially and temporally adjacent regions.

Key-segments for video object segmentation

The method first identifies object-like regions in any frame according to both static and dynamic cues and compute a series of binary partitions among candidate “key-segments” to discover hypothesis groups with persistent appearance and motion.

Fully Connected Object Proposals for Video Segmentation

A novel approach to video segmentation using multiple object proposals that combines appearance with long-range point tracks to ensure robustness with respect to fast motion and occlusions over longer video sequences is presented.

Video Segmentation via Object Flow

This work forms a principled, multiscale, spatio-temporal objective function that uses optical flow to propagate information between frames for video segmentation and demonstrates the effectiveness of jointly optimizing optical flow and video segmentations using an iterative scheme.

Primary Object Segmentation in Videos via Alternate Convex Optimization of Foreground and Background Distributions

The alternate convex optimization (ACO) scheme is developed, which decomposes the nonconvex optimization into two quadratic programs and proposes the forward-backward strategy, which performs the segmentation sequentially from the first to the last frames and then vice versa, to exploit temporal correlations.