Integrating domain knowledge: using hierarchies to improve deep classifiers

@article{Brust2019IntegratingDK,
  title={Integrating domain knowledge: using hierarchies to improve deep classifiers},
  author={Clemens-Alexander Brust and Joachim Denzler},
  journal={ArXiv},
  year={2019},
  volume={abs/1811.07125}
}
One of the most prominent problems in machine learning in the age of deep learning is the availability of sufficiently large annotated datasets. For specific domains, e.g. animal species, a long-tail distribution means that some classes are observed and annotated insufficiently. Additional labels can be prohibitively expensive, e.g. because domain experts need to be involved. However, there is more information available that is to the best of our knowledge not exploited accordingly. In this… 
No Cost Likelihood Manipulation at Test Time for Making Better Mistakes in Deep Networks
TLDR
It is found that current state-of-the-art hierarchy-aware deep classifiers do not always show practical improvement over the standard cross-entropy baseline in making better mistakes, and on examining mistake-severity distributions of the top-1 prediction, this work uses the classical Conditional Risk Minimization framework for hierarchy aware classification.
Computing Class Hierarchies from Classifiers
  • Kai Kang, Fangzhen Lin
  • Computer Science
    ArXiv
  • 2021
TLDR
This paper proposes a novel algorithm for automatically acquiring a class hierarchy from a classifier which is often a large neural network these days and produces surprisingly good hierarchies for some well-known deep neural network models trained on the CIFAR-10 dataset.
Leveraging Structural Cues for Better Training and Deployment in Computer Vision
With the growing use of computer vision tools in wide-ranging applications, it becomes imperative to understand and resolve issues in computer vision models when they are used in production settings
Making Every Label Count: Handling Semantic Imprecision by Integrating Domain Knowledge
TLDR
This work proposes CHILLAX (Class Hierarchies for Imprecise Label Learning and Annotation eXtrapolation), a method based on hierarchical classification, to fully utilize labels of any precision.
Self-Supervised Learning from Semantically Imprecise Data
TLDR
This work extends CH ChillAX with a self-supervised scheme using constrained extrapolation to generate pseudo-labels, enabling an even weaker supervision requirement than CHILLAX and is suitable as a drop-in replacement without any negative consequences such as longer training times.
United We Learn Better: Harvesting Learning Improvements From Class Hierarchies Across Tasks
TLDR
This work establishes a theoretical framework based on probability and set theory for extracting parent predictions and a hierarchical loss that can be used across tasks, showing results across classification and detection benchmarks and opening up the possibility of hierarchical learning for sigmoid-based detection architectures.
WikiChurches: A Fine-Grained Dataset of Architectural Styles with Real-World Challenges
TLDR
A novel dataset for architectural style classification, consisting of 9,485 images of church buildings, and provides 631 bounding box annotations of characteristic visual features for 139 churches from four major categories, to serve as a benchmark for various research fields.
Adaptive Knowledge Driven Regularization for Deep Neural Networks
TLDR
This paper derives CORR-Reg with the Bayesian inference framework and proposes a novel optimization algorithm with the Lagrange multiplier method and Stochastic Gradient Descent, which achieves significant improvement over state-of-the-art regularization methods.
NBDT: Neural-Backed Decision Tree
TLDR
This work improves accuracy and interpretability using Neural-Backed Decision Trees, a differentiable sequence of decisions and a surrogate loss that forces the model to learn high-level concepts and lessens reliance on highlyuncertain decisions.
Carpe Diem: A Lifelong Learning Tool for Automated Wildlife Surveillance
We introduce Carpe Diem, an interactive tool for object detection tasks such as automated wildlife surveillance. It reduces the annotation effort by automatically selecting informative images for
...
1
2
...

References

SHOWING 1-10 OF 53 REFERENCES
Transfer Learning in a Transductive Setting
TLDR
This work extends transfer learning with semi-supervised learning to exploit unlabeled instances of (novel) categories with no or only a few labeled instances to exploit the manifold structure of novel classes.
DeViSE: A Deep Visual-Semantic Embedding Model
TLDR
This paper presents a new deep visual-semantic embedding model trained to identify visual objects using both labeled image data as well as semantic information gleaned from unannotated text and shows that the semantic information can be exploited to make predictions about tens of thousands of image labels not observed during training.
Embedding Label Structures for Fine-Grained Feature Representation
TLDR
The proposed multitask learning framework significantly outperforms previous fine-grained feature representations for image retrieval at different levels of relevance and to model the multi-level relevance, label structures such as hierarchy or shared attributes are seamlessly embedded into the framework by generalizing the triplet loss.
Discriminative Transfer Learning with Tree-based Priors
TLDR
This work proposes a method for improving classification performance for high capacity classifiers by discovering similar classes and transferring knowledge among them, which learns to organize the classes into a tree hierarchy, and proposes an algorithm for learning the underlying tree structure.
Image Captioning and Visual Question Answering Based on Attributes and External Knowledge
TLDR
A visual question answering model that combines an internal representation of the content of an image with information extracted from a general knowledge base to answer a broad range of image-based questions and allows questions to be asked where the image alone does not contain the information required to select the appropriate answer.
LSDA: Large Scale Detection through Adaptation
TLDR
This paper proposes Large Scale Detection through Adaptation (LSDA), an algorithm which learns the difference between the two tasks and transfers this knowledge to classifiers for categories without bounding box annotated data, turning them into detectors.
Semantic Hierarchies for Visual Object Recognition
TLDR
The semantics of image labels are used to integrate prior knowledge about inter-class relationships into the visual appearance learning and to build and train a semantic hierarchy of discriminative classifiers and how to use it to perform object detection.
Hierarchy-Based Image Embeddings for Semantic Image Retrieval
TLDR
This work introduces a deterministic algorithm for computing the class centroids directly based on prior world-knowledge encoded in a hierarchy of classes such as WordNet, and shows that learned semantic image embeddings improve the semantic consistency of image retrieval results by a large margin.
Learning a Tree of Metrics with Disjoint Visual Features
TLDR
An approach to learn discriminative visual representations while exploiting external semantic knowledge about object category relationships is introduced, and a novel regularizer coupling the metrics that prefers a sparse disjoint set of features to be selected for each metric relative to its ancestor (supercategory) nodes' metrics is introduced.
Semantic Label Sharing for Learning with Many Categories
TLDR
A simple method of label sharing between semantically similar categories is proposed that leverages the WordNet hierarchy to define semantic distance between any two categories and use this semantic distance to share labels.
...
1
2
3
4
5
...