Super-Trajectory for Video Segmentation

  title={Super-Trajectory for Video Segmentation},
  author={Wenguan Wang and Jianbing Shen and Jianwen Xie and Fatih Murat Porikli},
  journal={2017 IEEE International Conference on Computer Vision (ICCV)},
We introduce a novel semi-supervised video segmentation approach based on an efficient video representation, called as “super-trajectory. [] Key Method We generate trajectories using a probabilistic model, which handles occlusions and drifts in a robust and natural way. To reliably group trajectories, we adopt a modified version of the density peaks based clustering algorithm that allows capturing rich spatiotemporal relations among trajectories in the clustering process.

Figures and Tables from this paper

Semi-Supervised Video Object Segmentation with Super-Trajectories
This work introduces a semi-supervised video segmentation approach based on an efficient video representation, called as “super-trajectory”, that is capable of extracting the target objects from complex backgrounds, and even reidentifying them after prolonged occlusions, producing high-quality video object segments.
Super-Trajectories: A Compact Yet Rich Video Representation
A new video representation in terms of an over-segmentation of dense trajectories covering the whole video that exploits constraints for edges in addition to trajectory based color and position similarity is proposed.
Automatic Video Object Segmentation Based on Visual and Motion Saliency
The proposed approach automatically segments the persistent foreground object meanwhile preserving the potential shape in videos, showing promising results on the challenging benchmark videos in comparison with the existing counterparts.
Effective online refinement for video object segmentation
A novel framework, which deeply explores the motion cue and the online fine-tuning strategy to tackle the task of semi-supervised video object segmentation is proposed, which achieves the state-of-the-art performance.
Object Discovery in Videos as Foreground Motion Clustering
A novel pixel-trajectory recurrent neural network that learns feature embeddings of foreground pixel trajectories linked across time is introduced that establishes correspondences between foreground object masks across video frames.
Weakly Supervised Video Object Segmentation
An energy minimization problem on a function that consists of unary term of object probability and pairwise terms of label smoothness potentials is solved to get the pixel-wise object segmentation mask of each frame.
DIPNet: Dynamic Identity Propagation Network for Video Object Segmentation
A Dynamic Identity Propagation Network (DIPNet) that adaptively propagates and accurately segments the video objects over time and provides state-of-the-art performance with time efficiency is proposed.
EpO-Net: Exploiting Geometric Constraints on Dense Trajectories for Motion Saliency
It is shown that by combining epipolar distances with optical flow, a powerful motion network can be learned and a robust fusion mechanism between motion and appearance-based features is proposed, which outperforms the previous methods on DAVIS-2016, 2017 and Segtrackv2 dataset.
Transparent object segmentation from casually captured videos
A from‐coarse‐to‐fine transparent object segmentation method, which utilizes trajectory clustering to roughly distinguish the transparent from the background and refine the segmentation based on combination information of color and distortion.


Video Object Segmentation Via Dense Trajectories
A new graph-based segmentation method is proposed which adopts both local and global motion information encoded by the tracked dense point trajectories, and achieves good performance on trajectory clustering and obtains accurate video object segmentation results on both the Moseg dataset and the new dataset containing more challenging videos.
Video segmentation by tracing discontinuities in a trajectory embedding
This work proposes a novel embedding discretization process that recovers from over-fragmentations by merging clusters according to discontinuity evidence along inter-cluster boundaries, and presents experimental results of the method that outperform the state-of-the-art in challenging motion segmentation datasets.
Multiple Hypothesis Video Segmentation from Superpixel Flows
This work determines the solution of this segmentation problem as the MAP labeling of a higher-order random field, and develops a framework that allows MHVS to achieve spatial and temporal long-range label consistency while operating in an on-line manner.
Track to the future: Spatio-temporal video segmentation with long-range motion cues
An efficient spatiotemporal video segmentation algorithm is developed, which naturally incorporates long-range motion cues from the past and future frames in the form of clusters of point tracks with coherent motion, and a new track clustering cost function is devised that includes occlusion reasoning, in the forms of depth ordering constraints, as well as motion similarity along the tracks.
Video object segmentation by tracking regions
A new circular dynamic-time warping (CDTW) algorithm is formulated that generalizes DTW to match closed boundaries of two regions, without compromising DTW's guarantees of achieving the optimal solution with linear complexity.
Track and Segment: An Iterative Unsupervised Approach for Video Object Proposals
  • Fanyi Xiao, Yong Jae Lee
  • Computer Science
    2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR)
  • 2016
This paper presents an unsupervised approach that generates a diverse, ranked set of bounding box and segmentation video object proposals-spatio-temporal tubes that localize the foreground objects-in an unannotated video, and demonstrates state-of-the-art segmentation results on the SegTrack v2 dataset.
Segmentation of Moving Objects by Long Term Video Analysis
This paper demonstrates that motion will be exploited most effectively, if it is regarded over larger time windows, and suggests working with a paradigm that starts with semi-dense motion cues first and that fills up textureless areas afterwards based on color.
Motion Trajectory Segmentation via Minimum Cost Multicuts
This paper provides a method to create a long-term point trajectory graph with attractive and repulsive binary terms and outperform state-of-the-art methods based on spectral clustering on the FBMS-59 dataset and on the motion subtask of the VSB100 dataset.
Fast Object Segmentation in Unconstrained Video
This method is fast, fully automatic, and makes minimal assumptions about the video, which enables handling essentially unconstrained settings, including rapidly moving background, arbitrary object motion and appearance, and non-rigid deformations and articulations.
Object Segmentation by Long Term Analysis of Point Trajectories
This paper presents a method that uses long term point trajectories based on dense optical flow to define pair-wise distances between these trajectories, which results in temporally consistent segmentations of moving objects in a video shot.