Hypothesis Testing and Machine Learning: Interpreting Variable Effects in Deep Artificial Neural Networks using Cohen's f2

@article{Messner2023HypothesisTA,
  title={Hypothesis Testing and Machine Learning: Interpreting Variable Effects in Deep Artificial Neural Networks using Cohen's f2},
  author={Wolfgang Messner},
  journal={ArXiv},
  year={2023},
  volume={abs/2302.01407}
}
  • W. Messner
  • Published 2 February 2023
  • Computer Science
  • ArXiv
Deep artificial neural networks show high predictive performance in many fields, but they do not afford statistical inferences and their black-box operations are too complicated for humans to comprehend. Because positing that a relationship exists is often more important than prediction in scientific experiments and research models, machine learning is far less frequently used than inferential statistics. Additionally, statistics calls for improving the test of theory by showing the magnitude… 
View PDF on arXiv

Figures and Tables from this paper

88 References

Interpreting Deep Neural Networks Through Variable Importance

This work proposes an effect size analogue for DNNs that is appropriate for applications with highly collinear predictors (ubiquitous in computer vision) and extends the recently proposed "RelATive cEntrality" (RATE) measure to the Bayesian deep learning setting.

Comparison of recent methods for inference of variable influence in neural networks

Stop ordering machine learning algorithms by their explainability! A user-centered investigation of performance and explainability

Machine Learning in Psychometrics and Psychological Research

It is claimed that complementing the analytical workflow of psychological experiments with Machine Learning-based analysis will both maximize accuracy and minimize replicability issues.

All Models are Wrong, but Many are Useful: Learning a Variable's Importance by Studying an Entire Class of Prediction Models Simultaneously

Model class reliance (MCR) is proposed as the range of VI values across all well-performing model in a prespecified class, which gives a more comprehensive description of importance by accounting for the fact that many prediction models, possibly of different parametric forms, may fit the data well.

Visualizing the effects of predictor variables in black box supervised learning models

    D. ApleyJingyu Zhu
    Computer Science
    Journal of the Royal Statistical Society: Series B (Statistical Methodology)
  • 2020
Accumulated local effects plots are presented, which do not require this unreliable extrapolation with correlated predictors and are far less computationally expensive than partial dependence plots.

Computer Age Statistical Inference: Algorithms, Evidence, and Data Science

This book takes an exhilarating journey through the revolution in data analysis following the introduction of electronic computation in the 1950s, with speculation on the future direction of statistics and data science.

Interpretable Machine Learning

This project introduces Robust T CAV, which builds on TCAV and experimentally determines best practices for this method and is a step in the direction of making TCAVs, an already impactful algorithm in interpretability, more reliable and useful for practitioners.

Assessing the contribution of variables in feed forward neural network

A novel method for examination of the variable contribution to computational neural network models

...