Taskology: Utilizing Task Relations at Scale

  title={Taskology: Utilizing Task Relations at Scale},
  author={Yao Lu and S{\"o}ren Pirk and Jan Dlabal and Anthony Brohan and Ankita Pasad and Zhao Chen and Vincent Casser and Anelia Angelova and A. Gordon},
  journal={2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)},
  • Yao Lu, S. Pirk, +6 authors A. Gordon
  • Published 14 May 2020
  • Computer Science
  • 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)
Many computer vision tasks address the problem of scene understanding and are naturally interrelated e.g. object classification, detection, scene segmentation, depth estimation, etc. We show that we can leverage the inherent relationships among collections of tasks, as they are trained jointly, supervising each other through their known relationships via consistency losses. Furthermore, explicitly utilizing the relationships between tasks allows improving their performance while dramatically… Expand

Figures and Tables from this paper

Semi-Supervised Learning for Multi-Task Scene Understanding by Neural Graph Consensus
It is shown how prediction of different representations such as depth, semantic segmentation, surface normals and pose from RGB input could be effectively learned through self-supervised consensus in a graph of neural networks. Expand
Single Image Depth Estimation: An Overview
This work reviews solutions to the problem of depth estimation, arguably the most important subtask in scene understanding, and examines non-deep learning approaches that mostly predate deep learning and utilize hand-crafted features and assumptions, and more recent works that mostly use deep learning techniques. Expand
Learning Multiple Dense Prediction Tasks from Partially Annotated Data
  • Wei-Hong Li, Xialei Liu, Hakan Bilen
  • Computer Science
  • 2021
Despite the recent advances in multi-task learning of dense prediction problems, most methods rely on expensive labelled datasets. In this paper, we present a label efficient approach and look atExpand
CARLS: Cross-platform Asynchronous Representation Learning System
The proposed CARLS is particularly suitable for learning paradigms where model training benefits from additional knowledge inferred or discovered during training, such as node embeddings for graph neural networks or reliable pseudo labels from model predictions. Expand


Taskonomy: Disentangling Task Transfer Learning
This work proposes a fully computational approach for modeling the structure of space of visual tasks via finding (first and higher-order) transfer learning dependencies across a dictionary of twenty six 2D, 2.5D, 3D, and semantic tasks in a latent space and provides a computational taxonomic map for task transfer learning. Expand
Multi-task Self-Supervised Visual Learning
The results show that deeper networks work better, and that combining tasks—even via a na¨ýve multihead architecture—always improves performance. Expand
Integrated perception with recurrent multi-task neural networks
This work proposes a new architecture, which it calls "MultiNet", in which not only deep image features are shared between tasks, but where tasks can interact in a recurrent manner by encoding the results of their analysis in a common shared representation of the data. Expand
Multi-task Learning with Labeled and Unlabeled Tasks
Focusing on an instance-based transfer method, this work derives an algorithm for making the choice of labeled tasks and for transferring information between the tasks in a principled way and illustrates the effectiveness of the algorithm by experiments on synthetic and real data. Expand
Multi-task Learning Using Uncertainty to Weigh Losses for Scene Geometry and Semantics
A principled approach to multi-task deep learning is proposed which weighs multiple loss functions by considering the homoscedastic uncertainty of each task, allowing us to simultaneously learn various quantities with different units or scales in both classification and regression settings. Expand
UberNet: Training a Universal Convolutional Neural Network for Low-, Mid-, and High-Level Vision Using Diverse Datasets and Limited Memory
  • I. Kokkinos
  • Computer Science
  • 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR)
  • 2017
In this work we train in an end-to-end manner a convolutional neural network (CNN) that jointly handles low-, mid-, and high-level vision tasks in a unified architecture. Such a network can act likeExpand
Representation Learning by Learning to Count
This paper uses two image transformations in the context of counting: scaling and tiling to train a neural network with a contrastive loss that produces representations that perform on par or exceed the state of the art in transfer learning benchmarks. Expand
Transitive Invariance for Self-Supervised Visual Representation Learning
This paper proposes to generate a graph with millions of objects mined from hundreds of thousands of videos and argues to organize and reason the data with multiple variations to exploit different self-supervised approaches to learn representations invariant to inter-instance variations. Expand
Unsupervised Visual Representation Learning by Context Prediction
It is demonstrated that the feature representation learned using this within-image context indeed captures visual similarity across images and allows us to perform unsupervised visual discovery of objects like cats, people, and even birds from the Pascal VOC 2011 detection dataset. Expand
Unsupervised Learning of Visual Representations by Solving Jigsaw Puzzles
A novel unsupervised learning approach to build features suitable for object detection and classification and to facilitate the transfer of features to other tasks, the context-free network (CFN), a siamese-ennead convolutional neural network is introduced. Expand