Amazon SageMaker Clarify: Machine Learning Bias Detection and Explainability in the Cloud

  title={Amazon SageMaker Clarify: Machine Learning Bias Detection and Explainability in the Cloud},
  author={Michaela Hardt and Xiaoguang Chen and Xiaoyi Cheng and Michele Donini and Jason Gelman and Satish Gollaprolu and John He and Pedro Larroy and Xinyu Liu and Nick McCarthy and Ashish M. Rathi and Scott Rees and Ankit Siva and ErhYuan Tsai and Keerthan Vasist and Pinar Yilmaz and Muhammad Bilal Zafar and Sanjiv Das and Kevin Haas and Tyler Hill and Krishnaram Kenthapadi},
  journal={Proceedings of the 27th ACM SIGKDD Conference on Knowledge Discovery \& Data Mining},
Understanding the predictions made by machine learning (ML) models and their potential biases remains a challenging and labor-intensive task that depends on the application, the dataset, and the specific model. We present Amazon SageMaker Clarify, an explainability feature for Amazon SageMaker that launched in December 2020, providing insights into data and ML models by identifying biases and explaining predictions. It is deeply integrated into Amazon SageMaker, a fully managed service that… 

Figures and Tables from this paper

Amazon SageMaker Model Monitor: A System for Real-Time Insights into Deployed Machine Learning Models

Amazon SageMaker Model Monitor is presented, a fully managed service that continuously monitors the quality of machine learning models hosted on Amazon SageMaker and automatically detects data, concept, bias, and feature attribution drift in models in real-time and provides alerts so that model owners can take corrective actions and thereby maintain high quality models.

A Human-Centric Take on Model Monitoring

It is found that relevant stakeholders would want model monitoring systems to provide clear, unambiguous, and easy-to-understand insights that are readily actionable, and that stakeholders desire customization of model Monitoring systems to cater to domain-specific use cases.

Model Monitoring in Practice: Lessons Learned and Open Challenges

This tutorial motivates the need for ML model monitoring, as part of a broader AI model governance and responsible AI framework, from societal, legal, customer/end-user, and model developer perspectives, and provides a roadmap for thinking about model monitoring in practice.

Mitigating Bias in Algorithmic Systems - A Fish-Eye View

The literature describes three steps toward a comprehensive treatment – bias detection, fairness management and explainability management – and underscores the need to work from within the system as well as from the perspective of stakeholders in the broader context.

Fairness in Recommendation: A Survey

This survey focuses on the foundations for fairness in recommendation literature with a focus on the taxonomies of current fairness definitions, the typical techniques for improving fairness, as well as the datasets for fairness studies in recommendation.

Fairness Testing: A Comprehensive Survey and Analysis of Trends

A comprehensive survey of existing research on fairness testing is provided, collecting 113 papers and analyzing the research focus, trends, promising directions, as well as widely-adopted datasets and open source tools for fairness testing.

Constructive Interpretability with CoLabel: Corroborative Integration, Complementary Features, and Collaborative Learning

CoLabel is presented, a construc-tively interpretable model that provides explanations that help in self-diagnosing and verifying predictions and achieves superior accuracy to the state-of-the-art black-box models.



Explainable machine learning in deployment

This study explores how organizations view and use explainability for stakeholder consumption, and synthesizes the limitations of current explainability techniques that hamper their use for end users.

Elastic Machine Learning Algorithms in Amazon SageMaker

The computational model behind Amazon SageMaker, which is an ML platform provided as part of Amazon Web Services, and supports incremental training, resumable and elastic learning as well as automatic hyperparameter optimization, is described.

The What-If Tool: Interactive Probing of Machine Learning Models

The What-If Tool is an open-source application that allows practitioners to probe, visualize, and analyze ML systems, with minimal coding, and lets practitioners measure systems according to multiple ML fairness metrics.

"Why Should I Trust You?": Explaining the Predictions of Any Classifier

LIME is proposed, a novel explanation technique that explains the predictions of any classifier in an interpretable and faithful manner, by learning aninterpretable model locally varound the prediction.

Improving Fairness in Machine Learning Systems: What Do Industry Practitioners Need?

This first systematic investigation of commercial product teams' challenges and needs for support in developing fairer ML systems identifies areas of alignment and disconnect between the challenges faced by teams in practice and the solutions proposed in the fair ML research literature.

Defuse: Harnessing Unrestricted Adversarial Examples for Debugging Models Beyond Test Accuracy

An algorithm inspired by adversarial machine learning techniques that uses a generative model to find naturally occurring instances misclassified by a model and proposes Defuse, a method that generates novel model misclassifications, categorizes these errors into high-level “model bugs”, and efficiently labels and finetunes on the errors to correct them.

LiFT: A Scalable Framework for Measuring Fairness in ML Applications

The LinkedIn Fairness Toolkit (LiFT) is presented, a framework for scalable computation of fairness metrics as part of large ML systems and the design of the fairness measurement system is presented.

Mitigating Unwanted Biases with Adversarial Learning

This work presents a framework for mitigating biases concerning demographic groups by including a variable for the group of interest and simultaneously learning a predictor and an adversary, which results in accurate predictions that exhibit less evidence of stereotyping Z.

A Unified Approach to Interpreting Model Predictions

A unified framework for interpreting predictions, SHAP (SHapley Additive exPlanations), which unifies six existing methods and presents new methods that show improved computational performance and/or better consistency with human intuition than previous approaches.

Faithful and Customizable Explanations of Black Box Models

Model Understanding through Subspace Explanations (MUSE), a novel model agnostic framework which facilitates understanding of a given black box model by explaining how it behaves in subspaces characterized by certain features of interest, is proposed.