Learning Policies for Adaptive Tracking with Deep Feature Cascades

  title={Learning Policies for Adaptive Tracking with Deep Feature Cascades},
  author={Chen Huang and Simon Lucey and Deva Ramanan},
  journal={2017 IEEE International Conference on Computer Vision (ICCV)},
Visual object tracking is a fundamental and time-critical vision task. Recent years have seen many shallow tracking methods based on real-time pixel-based correlation filters, as well as deep methods that have top performance but need a high-end GPU. In this paper, we learn to improve the speed of deep trackers without losing accuracy. Our fundamental insight is to take an adaptive approach, where easy frames are processed with cheap features (such as pixel values), while challenging frames are… 

Figures and Tables from this paper

Visual Tracking via Hierarchical Deep Reinforcement Learning
This work shows how to teach machines to track a generic object in videos like humans, who can use a few search steps to perform tracking, and builds a Markov decision process in Deep Reinforcement Learning (DRL) that agents can learn to determine hierarchical decisions on tracking mode and motion estimation.
Deep Reinforcement Learning with Iterative Shift for Visual Tracking
A deep reinforcement learning with iterative shift (DRL-IS) method for single object tracking, where an actor-critic network is introduced to predict the iterative shifts of object bounding boxes, and evaluate the shifts to take actions on whether to update object models or re-initialize tracking.
Unsupervised Deep Representation Learning for Real-Time Tracking
This work proposes an unsupervised learning method for visual tracking that achieves the baseline accuracy of classic fully supervised trackers while achieving a real-time speed and exhibits a potential in leveraging more unlabeled or weakly labeled data to further improve the tracking accuracy.
Hierarchical Tracking by Reinforcement Learning-Based Searching and Coarse-to-Fine Verifying
This work proposes a hierarchical tracker that learns to move and track based on the combination of data-driven search at the coarse level and coarse-to-fine verification at the fine level, and utilizes a recurrent convolutional neural network-based deep Q-network to effectively learn data- driven searching policies.
A Meta-Q-Learning Approach to Discriminative Correlation Filter based Visual Tracking
This study proposes Meta-Q-learning Correlation Filter (MQCF), a method for dynamically determining the learning rate of a baseline DCF-based tracker based on hand-crafted features of Histogram of Oriented Gradient (HOG), by means of reinforcement learning.
Visual Tracking With Long-Short Term Based Correlation Filter
A novel real-time long-short-term multi-model based tracking method designed to solve the inaccurate prediction of object position due to the cosine window in the correlation filter and the bounding-box regression strategy is introduced.
Visual Tracking by Means of Deep Reinforcement Learning and an Expert Demonstrator
A3CT, which exploits demonstrations of a state-of-the-art tracker to learn an effective tracking policy, and A3CTD, that takes advantage of the same expert tracker to correct its behaviour during tracking are proposed.
Robust adaptive learning with Siamese network architecture for visual tracking
A robust adaptive learning visual tracking algorithm that employs the average peak-to-correlation energy (APCE) to determinate whether to update the learning change factors to improve the accuracy of tracking model and reduce the tracking drift in the case of tracking failure, deformation or background blur.
HNST: Hybrid Neural State Tracker for High Speed Tracking
  • Zhe Zou, Yujie Wu, Rong Zhao
  • Computer Science
    2021 7th International Conference on Control, Automation and Robotics (ICCAR)
  • 2021
This work proposes a hybrid neural state tracker (HNST) that combines DNN-based detection and Kernelized Correlation Filter tracking using a policy to determine when to activate detection and whether to update the template of tracking target, which indicates the great potential of HNST in high-speed tracking.


Deep Reinforcement Learning for Visual Object Tracking in Videos
This paper introduces a fully end-to-end approach for visual tracking in videos that learns to predict the bounding box locations of a target object at every frame and is the first neural-network tracker that combines convolutional and recurrent networks with RL algorithms.
Learning to Track at 100 FPS with Deep Regression Networks
This work proposes a method for offline training of neural networks that can track novel objects at test-time at 100 fps, which is significantly faster than previous methods that use neural networks for tracking, which are typically very slow to run and not practical for real-time applications.
Visual Tracking by Reinforced Decision Making
This paper introduces a novel visual tracking algorithm based on a template selection strategy constructed by deep reinforcement learning methods and shows that the tracking algorithm effectively decides the best template for visual tracking.
Visual Tracking with Fully Convolutional Networks
An in-depth study on the properties of CNN features offline pre-trained on massive image data and classification task on ImageNet shows that the proposed tacker outperforms the state-of-the-art significantly.
Incremental Learning for Robust Visual Tracking
A tracking method that incrementally learns a low-dimensional subspace representation, efficiently adapting online to changes in the appearance of the target, and includes a method for correctly updating the sample mean and a forgetting factor to ensure less modeling power is expended fitting older observations.
Transferring Rich Feature Hierarchies for Robust Visual Tracking
This work pre-training a CNN offline and then transferring the rich feature hierarchies learned to online tracking, and proposes to generate a probability map instead of producing a simple class label to fit the characteristics of object tracking.
Hierarchical Convolutional Features for Visual Tracking
This paper adaptively learn correlation filters on each convolutional layer to encode the target appearance and hierarchically infer the maximum response of each layer to locate targets.
Struck: Structured Output Tracking with Kernels
A framework for adaptive visual object tracking based on structured output prediction that is able to outperform state-of-the-art trackers on various benchmark videos and can easily incorporate additional features and kernels into the framework, which results in increased tracking performance.
Recurrently Target-Attending Tracking
A novel tracking method called Recurrently Target-attending Tracking (RTT), which attempts to identify and exploit those reliable parts which are beneficial for the overall tracking process and derives an efficient closedform solution with a sharp reduction in computation complexity.
Fully-Convolutional Siamese Networks for Object Tracking
A basic tracking algorithm is equipped with a novel fully-convolutional Siamese network trained end-to-end on the ILSVRC15 dataset for object detection in video and achieves state-of-the-art performance in multiple benchmarks.