Unremarkable AI: Fitting Intelligent Decision Support into Critical, Clinical Decision-Making Processes

  title={Unremarkable AI: Fitting Intelligent Decision Support into Critical, Clinical Decision-Making Processes},
  author={Qian Yang and Aaron Steinfeld and John Zimmerman},
  journal={Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems},
Clinical decision support tools (DST) promise improved healthcare outcomes by offering data-driven insights. While effective in lab settings, almost all DSTs have failed in practice. Empirical research diagnosed poor contextual fit as the cause. This paper describes the design and field evaluation of a radically new form of DST. It automatically generates slides for clinicians' decision meetings with subtly embedded machine prognostics. This design took inspiration from the notion of… 

Figures from this paper

To explain or not to explain?—Artificial intelligence explainability in clinical decision support systems

Whether explainability can provide added value to CDSS depends on several key questions: technical feasibility, the level of validation in case of explainable algorithms, the characteristics of the context in which the system is implemented, the designated role in the decision-making process, and the key user group(s).

Clinical Evidence Engine: Proof-of-Concept For A Clinical-Domain-Agnostic Decision Support Infrastructure

The idea of designing DST explanations not as specific to a DST or an algorithm, but as a domain-agnostic decision support infrastructure is discussed, which opens up the possibility of enabling clinicians to rapidly determine the match between a clinical trial and a clinical question, and understand the result and contexts of the trial without extensive reading.

Get To The Point! Problem-Based Curated Data Views To Augment Care For Critically Ill Patients

A novel approach to designing CDSSs that are embedded in clinical workflows, by presenting problem-based curated data views tailored for problem-driven discovery, team communication, and situational awareness is introduced.

"Hello AI": Uncovering the Onboarding Needs of Medical Practitioners for Human-AI Collaborative Decision-Making

This work investigates the key types of information medical experts desire when they are first introduced to a diagnostic AI assistant, providing a richer understanding of what experts find important in their introduction to AI assistants before integrating them into routine practice.

“Brilliant AI Doctor” in Rural Clinics: Challenges in AI-Powered Clinical Decision Support System Deployment

The various tensions between the design of an AI-CDSS system (“Brilliant Doctor”) and the rural clinical context, such as the misalignment with local context and workflow, the technical limitations and usability barriers, as well as issues related to transparency and trustworthiness of AI- CDSS are reported.

Inclusion of Clinicians in the Development and Evaluation of Clinical Artificial Intelligence Tools: A Systematic Literature Review

To realize the full potential of AI tools within healthcare settings, this review suggests there are opportunities to more thoroughly integrate frontline users’ needs and feedback in the design process.

Assessing the communication gap between AI models and healthcare professionals: explainability, utility and trust in AI-driven clinical decision-making

This paper contributes with a pragmatic evaluation framework for explainable Machine Learning (ML) models for clinical decision support. The study revealed a more nuanced role for ML explanation

Lessons Learned from Designing an AI-Enabled Diagnosis Tool for Pathologists

A series of collaborative techniques to engage human pathologists with AI given AI's capabilities and limitations are proposed, based on which is prototype Impetus --- a tool where an AI takes various degrees of initiatives to provide various forms of assistance to a pathologist in detecting tumors from histological slides.

Two Case Studies of Experience Prototyping Machine Learning Systems in the Wild

This workshop shares two projects as case studies in which people engage with ML in much more complicated and nuanced ways than the technical HCML work might assume, to start a discussion about such previously-unknown complexities and nuances of machine learning.

Validating a Clinical Decision Support System for Palliative Care using healthcare professionals' insights

This study validated a Palliative Care CDSS, The Aleph, through a user-centred methodology, considering the predictions of the AI core, the usability, and the user experience, and identified most of the workflow implementation issues.



Investigating the Heart Pump Implant Decision Process: Opportunities for Decision Support Tools to Help

A field study investigating how clinicians make a heart pump implant decision with a focus on how to best integrate an intelligent DST into their work process reveals a lack of perceived need for and trust of machine intelligence, as well as many barriers to computer use at the point of clinical decision-making.

Commentary: Prognostic models: clinically useful or quickly forgotten?

The main reasons why doctors reject published prognostic models are lack of clinical credibility and lack of evidence that a prognostic model can support decisions about patient care (that is, evidence of accuracy, generality, and effectiveness).

Clinical decision support systems.

In the near future, the specificity of alerts will be improved, notifications will be prioritised and offer detailed advice, customisation of CDS will play an increasing role, and finally, CDS is heading for patient-centred decision support.

Clinical decision-support systems

From the earliest days of computing people have recognized that computers might support physicians by helping these people to sift through the vast collection of possible diseases and symptoms, and this idea has been echoed in futuristic works of science fiction.

“Many miles to go …”: a systematic review of the implementation of patient decision support interventions into routine clinical practice

The results point to significant challenges to the implementation of patient decision support using a referral model, including indifference on the part of health care professionals, and the lack of incentives that reward the use of these interventions needs to be considered as a significant impediment.

More Text Please! Understanding and Supporting the Use of Visualization for Clinical Text Overview

This work presents an investigation into the role and use of text in clinical practice, and reports on efforts to assess the best of both worlds---text and visualization---to facilitate clinical overview.

Improving clinical practice using clinical decision support systems: a systematic review of trials to identify features critical to success

Clinicians and other stakeholders should implement clinical decision support systems that incorporate these features whenever feasible and appropriate.

Effects of clinical decision-support systems on practitioner performance and patient outcomes: a synthesis of high-quality systematic review findings

There is significant evidence that CDSS can positively impact healthcare providers' performance with drug ordering and preventive care reminder systems as most clear examples.

Barriers and facilitators to implementing shared decision-making in clinical practice: a systematic review of health professionals' perceptions

It is revealed that interventions to foster implementation of shared decision-making in clinical practice will need to address a broad range of factors, and that on this subject there is very little known about any health professionals others than physicians.

Clinical evaluations and collaborative design: developing new technologies for mental healthcare interventions

This evaluation has provided strong initial evidence that games such as PI offer the potential to improve adolescent engagement in talk-based interventions and provided an insight into factors which should be considered in future designs in the MHC domain, e.g. the need to incorporate high levels of adaptability in future systems.