Adaptive ROI Generation for Video Object Segmentation Using Reinforcement Learning

  title={Adaptive ROI Generation for Video Object Segmentation Using Reinforcement Learning},
  author={Mingjie Sun and Jimin Xiao and Eng Gee Lim and Yanchun Xie and Jiashi Feng},
  journal={Pattern Recognit.},
In this paper, we aim to tackle the task of semi-supervised video object segmentation across a sequence of frames where only the ground-truth segmentation of the first frame is provided. The challenges lie in how to online update the segmentation model initialized from the first frame adaptively and accurately, even in presence of multiple confusing instances or large object motion. The existing approaches rely on selecting the region of interest for model update, which however, is rough and… Expand
Fast Pixel-Matching for Video Object Segmentation
A model, called NPMCA-net, is proposed, which directly localizes foreground objects based on mask-propagation and non-local technique by matching pixels in reference and target frames, which is robust to large object appearance variation, and can better adapt to occlusions. Expand
Deep learning for liver tumour classification: enhanced loss function
The study solved the problem in linear mapping of support vector machine and enhanced the classification accuracy and the processing time of early diagnosis of three different types of tumours in liver MRI images. Expand
Iterative Shrinking for Referring Expression Grounding Using Deep Reinforcement Learning
This paper proposes an iterative shrinking mechanism to localize the target, where the shrinking direction is decided by a reinforcement learning agent, with all contents within the current image patch comprehensively considered. Expand
Learning to Recommend Frame for Interactive Video Object Segmentation in the Wild
The frame selection problem in the interactive VOS is formulated as a Markov Decision Process, where an agent is learned to recommend the frame under a deep reinforcement learning framework, making the interactive setting more practical in the wild. Expand


Online Adaptation of Convolutional Neural Networks for Video Object Segmentation
Online Adaptive Video Object Segmentation (OnAVOS) is proposed which updates the network online using training examples selected based on the confidence of the network and the spatial configuration and adds a pretraining step based on objectness, which is learned on PASCAL. Expand
Learning Video Object Segmentation from Static Images
It is demonstrated that highly accurate object segmentation in videos can be enabled by using a convolutional neural network (convnet) trained with static images only, and a combination of offline and online learning strategies are used. Expand
Reinforcement Cutting-Agent Learning for Video Object Segmentation
This paper forms this problem as a Markov Decision Process, where agents are learned to segment object regions under a deep reinforcement learning framework, and establishes a novel reinforcement cutting-agent learning framework that achieves outperforming VOS performance on two public benchmarks. Expand
Online Meta Adaptation for Fast Video Object Segmentation
Conventional deep neural networks based video object segmentation (VOS) methods are dominated by heavily fine-tuning a segmentation model on the first frame of a given video, which is time-consumingExpand
Fast and Accurate Online Video Object Segmentation via Tracking Parts
This paper proposes a fast and accurate video object segmentation algorithm that can immediately start the segmentation process once receiving the images, and performs favorably against state-of-the-art algorithms in accuracy on the DAVIS benchmark dataset, while achieving much faster runtime performance. Expand
MoNet: Deep Motion Exploitation for Video Object Segmentation
A novel MoNet model to deeply exploit motion cues for boosting video object segmentation performance from two aspects, i.e., frame representation learning and segmentation refinement, provides new state-of-the-art performance on three competitive benchmark datasets. Expand
Blazingly Fast Video Object Segmentation with Pixel-Wise Metric Learning
The proposed method supports different kinds of user input such as segmentation mask in the first frame (semi-supervised scenario), or a sparse set of clicked points (interactive scenario), and reaches comparable quality to competing methods with much less interaction. Expand
Video Segmentation by Tracking Many Figure-Ground Segments
An unsupervised video segmentation approach by simultaneously tracking multiple holistic figure-ground segments that outperforms state-of-the-art approaches in the dataset, showing its efficiency and robustness to challenges in different video sequences. Expand
Fast Video Object Segmentation by Reference-Guided Mask Propagation
A deep Siamese encoder-decoder network is proposed that is designed to take advantage of mask propagation and object detection while avoiding the weaknesses of both approaches, and achieves accuracy competitive with state-of-the-art methods while running in a fraction of time compared to others. Expand
Video Object Segmentation without Temporal Information
Semantic One-Shot Video Object Segmentation is presented, based on a fully-convolutional neural network architecture that is able to successively transfer generic semantic information, learned on ImageNet, to the task of foreground segmentation, and finally to learning the appearance of a single annotated object of the test sequence (hence one shot). Expand