Efficient estimation of the ANOVA mean dimension, with an application to neural net classification

@article{Hoyt2021EfficientEO,
  title={Efficient estimation of the ANOVA mean dimension, with an application to neural net classification},
  author={Christopher R. Hoyt and Art B. Owen},
  journal={SIAM/ASA J. Uncertain. Quantification},
  year={2021},
  volume={9},
  pages={708-730}
}
The mean dimension of a black box function of $d$ variables is a convenient way to summarize the extent to which it is dominated by high or low order interactions. It is expressed in terms of $2^d-1$ variance components but it can be written as the sum of $d$ Sobol' indices that can be estimated by leave one out methods. We compare the variance of these leave one out methods: a Gibbs sampler called winding stairs, a radial sampler that changes each variable one at a time from a baseline, and a… 

Figures and Tables from this paper

The Mean Dimension of Neural Networks - What causes the interaction effects?

TLDR
This work proposes an estimation procedure that allows the calculation of the mean dimension from a given dataset, without resampling from external distributions, and shows that this variant possesses the zero independence property.

Global sensitivity analysis in epidemiological modeling

References

SHOWING 1-10 OF 36 REFERENCES

Estimating Mean Dimensionality of Analysis of Variance Decompositions

Analysis of variance (ANOVA) is now often applied to functions defined on the unit cube, where it serves as a tool for the exploratory analysis of functions. The mean dimension of a function, defined

Generalized Functional ANOVA Diagnostics for High-Dimensional Functions of Dependent Variables

TLDR
A weighted functional ANOVA that controls for the effect of dependence between input variables and is demonstrated in the context of machine learning in which the possibility of poor extrapolation makes it important to restrict attention to regions of high data density.

THE DIMENSION DISTRIBUTION AND QUADRATURE TEST FUNCTIONS

This paper introduces the dimension distribution for a square integrable function f on (0; 1) s . The dimension distribution is used to relate several denitions of the eectiv e dimension of a

Generalized Sobol sensitivity indices for dependent variables: numerical methods

The hierarchically orthogonal functional decomposition of any measurable function η of a random vector X=(X1, … , Xp) consists in decomposing η(X) into a sum of increasing dimension functions

Making best use of model evaluations to compute sensitivity indices

Mean Dimension of Ridge Functions

TLDR
It is shown that preintegrating a ridge function yields a new, potentially much smoother ridge function, and an example where, if one of the ridge coefficients is bounded away from zero as $d\to\infty$, then preintegration can reduce the mean dimension from $O(\sqrt{d})$ to $O(1)$.

A Review on Global Sensitivity Analysis Methods

This chapter makes a review, in a complete methodological framework, of various global sensitivity analysis methods of model output. Numerous statistical and probabilistic tools (regression,

An approximation theoretic perspective of the Sobol' indices with dependent variables.

The Sobol' indices are a recognized tool in global sensitivity analysis. When the uncertain variables in a model are statistically independent, the Sobol' indices may be easily interpreted and