Opening up Open World Tracking

  title={Opening up Open World Tracking},
  author={Yang Liu and Idil Esen Zulfikar and Jonathon Luiten and Achal Dave and Aljosa Osep and Deva Ramanan and Bastian Leibe and Laura Leal-Taix{\'e}},
  journal={2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)},
Tracking and detecting any object, including ones never-seen-before during model training, is a crucial but elusive capability of autonomous systems. An autonomous agent that is blind to never-seen-before objects poses a safety hazard when operating in the real world - and yet this is how almost all current systems work. One of the main obstacles towards advancing tracking any object is that this task is notoriously difficult to evaluate. A benchmark that would allow us to perform an apples-to… 

Visual Object Tracking with Discriminative Filters and Siamese Networks: A Survey and Outlook

This survey presents a systematic and thorough review of more than 90 DCFs and Siamese trackers, based on results in nine tracking benchmarks, and distinguishes and comprehensively review the shared as well as specific open research challenges in both these tracking paradigms.

Tracking Every Thing in the Wild

A new metric, Track Every Thing Accuracy (TETA), is introduced, breaking tracking measurement into three sub-factors: localization, association, and classification, allowing comprehensive benchmarking of tracking performance even under inaccurate classification.

BURST: A Benchmark for Unifying Object Recognition, Segmentation and Tracking in Video

This paper proposes BURST, a dataset which contains thousands of diverse videos with high-quality object masks, and an associated benchmark with six tasks involving object tracking and segmentation in video, which enables researchers to consider them in unison, and hence, more effectively pool knowledge from different methods across different tasks.



TAO: A Large-Scale Benchmark for Tracking Any Object

It is shown that existing single- and multi-object trackers struggle when applied to this scenario in the wild, and that detection-based, multi- object trackers are in fact competitive with user-initialized ones.

Exploiting Spatial Invariance for Scalable Unsupervised Object Tracking

An architecture that scales well to the large-scene, many-object setting by employing spatially invariant computations (convolutions and spatial attention) and representations (a spatially local object specification scheme) is proposed.

Tracking Without Bells and Whistles

Overall, Tracktor yields superior tracking performance than any current tracking method and the analysis exposes remaining and unsolved tracking challenges to inspire future research directions.

MOTChallenge: A Benchmark for Single-camera Multiple Target Tracking

This paper collects the first three releases of the MOTChallenge and provides a categorization of state-of-the-art trackers and a broad error analysis, to help newcomers understand the related work and research trends in the MOT community, and hopefully shed some light into potential future research directions.

Joint tracking and segmentation of multiple targets

This work proposes a multi-target tracker that exploits low level image information and associates every (super)-pixel to a specific target or classifies it as background and obtains a video segmentation in addition to the classical bounding-box representation in unconstrained, real-world videos.

Simultaneous Detection and Tracking with Motion Modelling for Multiple Object Tracking

This work introduces Deep Motion Modeling Network (DMM-Net) that can estimate multiple objects' motion parameters to perform joint detection and association in an end-to-end manner and demonstrates the suitability of Omni-MOT for deep learning with DMMNet and makes the source code of the network public.

Towards Segmenting Anything That Moves

This work proposes a simple learning-based approach for spatio-temporal grouping that leverages motion cues from optical flow as a bottom-up signal for separating objects from each other, and shows that this model matches top-down methods on common categories, while significantly out-performing both top- down and bottom- up methods on never-before-seen categories.

Coupled Object Detection and Tracking from Static Cameras and Moving Vehicles

We present a novel approach for multi-object tracking which considers object detection and spacetime trajectory estimation as a coupled optimization problem. Our approach is formulated in a minimum

Unidentified Video Objects: A Benchmark for Dense, Open-World Segmentation

UVO (Unidentified Video Objects), a new benchmark for openworld class-agnostic object segmentation in videos, is presented and it is demonstrated that UVO can be used for other applications, such as object tracking and super-voxel segmentation.

Track, Then Decide: Category-Agnostic Vision-Based Multi-Object Tracking

This paper presents an efficient segmentation mask-based tracker which associates pixel-precise masks reported by the segmentation with the capability to track generic unknown objects in the absence of such information.