• Corpus ID: 218870224

Statistical Analysis of Data Repeatability Measures

@article{Wang2020StatisticalAO,
  title={Statistical Analysis of Data Repeatability Measures},
  author={Zeyi Wang and Eric W. Bridgeford and Shan Wei Wang and Joshua T. Vogelstein and Brian S. Caffo},
  journal={arXiv: Applications},
  year={2020}
}
The advent of modern data collection and processing techniques has seen the size, scale, and complexity of data grow exponentially. A seminal step in leveraging these rich datasets for downstream inference is understanding the characteristics of the data which are repeatable -- the aspects of the data that are able to be identified under a duplicated analysis. Conflictingly, the utility of traditional repeatability measures, such as the intraclass correlation coefficient, under these settings… 

Figures from this paper

A Guide for Quantifying and Optimizing Measurement Reliability for the Study of Individual Differences

A two-dimensional individual variation field map is defined to decipher sources of individual variation and their relation to fingerprinting and measures of reliability and is proposed to use for dissecting within- and between-individual variation.

Neuroimaging brain growth charts: A road to mental health

The relationship between mental disorders and atypical brain development from a perspective of normative brain development is reviewed by surveying the recent progress in the development of brain growth charts, including four aspects on growth chart utility.

Toward Reliable Network Neuroscience for Mapping Individual Differences

This work aims to provide a benchmark for best practices by systematically comparing the reliability of human brain network measurements of individual differences under different analytical strategies using the test-retest design of the resting-state functional magnetic resonance imaging from the Human Connectome Project.

Eliminating accidental deviations to minimize generalization error: applications in connectomics and genomics

It is suggested that designing experiments and analyses to optimize discriminability may be a crucial step in solving the reproducibility crisis, and more generally, mitigating accidental measurement error.

On statistical tests of functional connectome fingerprinting

It is shown that a nearly universal Poisson approximation applies for different matching schemes, and theoretically investigate the permutation tests and explore the issue that the test is overly sensitive to uninteresting directions in the alternative hypothesis, such as clustering due to familial status or demographics.

References

SHOWING 1-10 OF 50 REFERENCES

Multilevel Functional Principal Component Analysis for High-Dimensional Data

The proposed inferential methods do not require loading the entire dataset at once in the computer memory and instead use only sequential access to data, which allows deployment of the methodology on low-resource computers where computations can be done in minutes on extremely large datasets.

Optimal Experimental Design for Big Data: Applications in Brain Imaging

An approach to experimental design that leverages multiple measurements for each distinct item, and introduces both a non-parameteric and parametric statistic to quantify the discriminability of potentially multivariate or non-Euclidean datasets.

Adjusting batch effects in microarray expression data using empirical Bayes methods.

Non-biological experimental variation or "batch effects" are commonly observed across multiple batches of microarray experiments, often rendering the task of combining data from these batches

Quantifying the reliability of image replication studies: The image intraclass correlation coefficient (I2C2)

The I2C2 generalizes the classic intraclass correlation (ICC) coefficient to the case when the data of interest are images, thereby providing a measure that is both intuitive and convenient in high-dimensional imaging studies.

Reproducibility of Single-Subject Functional Connectivity Measurements

An individual subject and control population converged to reliable different functional connectivity profiles that were task-modulated and could be discriminated with sufficient imaging time.

On statistical tests of functional connectome fingerprinting

It is shown that a nearly universal Poisson approximation applies for different matching schemes, and theoretically investigate the permutation tests and explore the issue that the test is overly sensitive to uninteresting directions in the alternative hypothesis, such as clustering due to familial status or demographics.

Factors affecting characterization and localization of interindividual differences in functional connectivity using MRI

A non‐parametric statistical metric is utilized that quantifies the degree to which a parameter set allows this individual subject differentiation by both maximizing interindividual variance and minimizing intra‐individual variance.

Assessing intrarater, interrater and test–retest reliability of continuous measurements

It is argued that the usual notion of product-moment correlation is well adapted in a test-retest situation, whereas the concept of intraclass correlation should be used for intrarater and interrater reliability.

Two simple approximations to the distributions of quadratic forms.

  • K. YuanP. Bentler
  • Mathematics
    The British journal of mathematical and statistical psychology
  • 2010
The paper systematically studies the quality of the two approximations and examines the effect of the lambda(i) and the degrees of freedom d by analysis and Monte Carlo, implying that the adjusted distribution for T can be as good as knowing its exact distribution.