Decision Rule Elicitation for Domain Adaptation

  title={Decision Rule Elicitation for Domain Adaptation},
  author={Alexander Nikitin and Samuel Kaski},
  journal={26th International Conference on Intelligent User Interfaces},
  • A. Nikitin, S. Kaski
  • Published 23 February 2021
  • Computer Science
  • 26th International Conference on Intelligent User Interfaces
Human-in-the-loop machine learning is widely used in artificial intelligence (AI) to elicit labels for data points from experts or to provide feedback on how close the predicted results are to the target. This simplifies away all the details of the decision-making process of the expert. In this work, we allow the experts to additionally produce decision rules describing their decision-making; the rules are expected to be imperfect but to give additional information. In particular, the rules can… 

Figures and Tables from this paper

Human-in-the-Loop Large-Scale Predictive Maintenance of Workstations
This work proposes a human-in-the-loop PdM approach in which a machine learning system predicts future problems in sets of workstations and interacts with domain experts to improve predictions and elicit their knowledge.
Interactive Decision Tree Learning and Decision Rule Extraction Based on the ImbTreeEntropy and ImbTreeAUC Packages
This paper presents two new R packages ImbTreeEntropy and ImbTreeAUC for building decision trees, including their interactive construction and analysis, which is a highly regarded feature for field


Where to Add Actions in Human-in-the-Loop Reinforcement Learning
This work proposes Expected Local Improvement (ELI), an automated method which selects states at which to query humans for a new action, and finds ELI demonstrates excellent empirical performance, even in settings where the synthetic "experts" are quite poor.
Knowledge elicitation via sequential probabilistic inference for high-dimensional prediction
This work proposes an algorithm and computational approximation for fast and efficient interaction, which sequentially identifies the most informative features on which to query expert knowledge to improve predictions in sparse linear regression.
Lifelong Machine Learning
  • Zhiyuan Chen, B. Liu
  • Computer Science, Education
    Synthesis Lectures on Artificial Intelligence and Machine Learning
  • 2016
As statistical machine learning matures, it is time to make a major effort to break the isolated learning tradition and to study lifelong learning to bring machine learning to new heights.
Power to the People: The Role of Humans in Interactive Machine Learning
It is argued that the design process for interactive machine learning systems should involve users at all stages: explorations that reveal human interaction patterns and inspire novel interaction methods, as well as refinement stages to tune details of the interface and choose among alternatives.
Interactive Prior Elicitation of Feature Similarities for Small Sample Size Prediction
Evaluation of the method in experiments with simulated and real users on text data confirm that prior elicitation of feature similarities improves prediction accuracy and elicitation with an interactive scatterplot display outperforms straightforward elicitation where the users choose feature pairs from a feature list.
Improving genomics-based predictions for precision medicine through active elicitation of expert knowledge
A probabilistic framework to incorporate expert feedback about the impact of genomic measurements on the outcome of interest is introduced and a novel approach to collect the feedback efficiently is presented, based on Bayesian experimental design.
Naive and Yet Enlightened: From Natural Frequencies to Fast and Frugal Decision Trees
An evolutionary view of rationality as an adaptive toolbox of fast and frugal heuristics is sometimes placed in opposition to probability as the ideal of enlightened rational human inference. Indeed,
Biographies, Bollywood, Boom-boxes and Blenders: Domain Adaptation for Sentiment Classification
This work extends to sentiment classification the recently-proposed structural correspondence learning (SCL) algorithm, reducing the relative error due to adaptation between domains by an average of 30% over the original SCL algorithm and 46% over a supervised baseline.
The Strength of Weak Learnability
In this paper, a method is described for converting a weak learning algorithm into one that achieves arbitrarily high accuracy, and it is shown that these two notions of learnability are equivalent.
Cooperative Inverse Reinforcement Learning
It is shown that computing optimal joint policies in CIRL games can be reduced to solving a POMDP, it is proved that optimality in isolation is suboptimal in C IRL, and an approximate CirL algorithm is derived.