Corpus ID: 90260109

Interpreting Black Box Models with Statistical Guarantees

  title={Interpreting Black Box Models with Statistical Guarantees},
  author={Collin Burns and Jesse Thomason and Wesley Tansey},
While many methods for interpreting machine learning models have been proposed, they are frequently ad hoc, difficult to evaluate, and come with no statistical guarantees on the error rate. This is especially problematic in scientific domains, where interpretations must be accurate and reliable. In this paper, we cast black box model interpretation as a hypothesis testing problem. The task is to discover "important" features by testing whether the model prediction is significantly different… Expand
Computationally Efficient Feature Significance and Importance for Machine Learning Models
A simple and computationally efficient significance test for the features of a machine learning model that identifies the statistically significant features as well as feature interactions of any order in a hierarchical manner, and generates a model-free notion of feature importance. Expand
Detecting and Understanding Generalization Barriers for Neural Machine Translation
This paper attempts to identify and understand generalization barrier words within an unseen input sentence that cause the degradation of fine-grained generalization in search-aware risk estimation through counterfactual generation. Expand
SurvNAM: The machine learning survival model explanation
A new modification of the Neural Additive Model (NAM) called SurvNAM and its modifications are proposed to explain predictions of the black-box machine learning survival model by using the Lasso-based regularization for functions from GAM and for a special representation of the GAM functions using their weighted linear and non-linear parts. Expand
SurvLIME: A method for explaining machine learning survival models
A new method called SurvLIME for explaining machine learning survival models is proposed, which is to apply the Cox proportional hazards model to approximate the survival model at the local area around a test example. Expand
An explanation method for Siamese neural networks
A new method for explaining the Siamese neural network using an autoencoder trained in a special way in order to take into account the embedding level of the Si-amese network, and its decoder part is used for reconstructing input data with the corresponding changes. Expand


The Holdout Randomization Test: Principled and Easy Black Box Feature Selection
The holdout randomization test (HRT) is proposed as a principled approach to feature selection using black box predictive models and produces a valid p-value for each feature, enabling control over the false discovery rate (or Type I error) for any predictive model. Expand
"Why Should I Trust You?": Explaining the Predictions of Any Classifier
LIME is proposed, a novel explanation technique that explains the predictions of any classifier in an interpretable and faithful manner, by learning aninterpretable model locally varound the prediction. Expand
A Unified Approach to Interpreting Model Predictions
A unified framework for interpreting predictions, SHAP (SHapley Additive exPlanations), which unifies six existing methods and presents new methods that show improved computational performance and/or better consistency with human intuition than previous approaches. Expand
Controlling the false discovery rate via knockoffs
In many fields of science, we observe a response variable together with a large number of potential explanatory variables, and would like to be able to discover which variables are truly associatedExpand
Interpretable Explanations of Black Boxes by Meaningful Perturbation
  • Ruth C. Fong, A. Vedaldi
  • Computer Science, Mathematics
  • 2017 IEEE International Conference on Computer Vision (ICCV)
  • 2017
A general framework for learning different kinds of explanations for any black box algorithm is proposed and the framework to find the part of an image most responsible for a classifier decision is specialised. Expand
Controlling the false discovery rate: a practical and powerful approach to multiple testing
SUMMARY The common approach to the multiplicity problem calls for controlling the familywise error rate (FWER). This approach, though, has faults, and we point out a few. A different approach toExpand
Panning for Gold: Model-X Knockoffs for High-dimensional Controlled Variable Selection
A new framework of model-X knockoffs is proposed, which reads from a different perspective the knockoff procedure, originally designed for controlling the false discovery rate in linear models, and demonstrates the superior power of knockoffs through simulations. Expand
Robust inference with knockoffs
We consider the variable selection problem, which seeks to identify important variables influencing a response $Y$ out of many candidate features $X_1, \ldots, X_p$. We wish to do so while offeringExpand
The conditional permutation test
We propose a general new method, the \emph{conditional permutation test}, for testing the conditional independence of variables $X$ and $Y$ given a potentially high-dimensional random vector $Z$ thatExpand
Learning to Explain: An Information-Theoretic Perspective on Model Interpretation
An efficient variational approximation to the mutual information is developed, and the effectiveness of the method is shown on a variety of synthetic and real data sets using both quantitative metrics and human evaluation. Expand