Numerically stable, single-pass, parallel statistics algorithms

@article{Bennett2009NumericallySS,
  title={Numerically stable, single-pass, parallel statistics algorithms},
  author={Janine Bennett and R. Grout and P. P{\'e}bay and D. Roe and D. Thompson},
  journal={2009 IEEE International Conference on Cluster Computing and Workshops},
  year={2009},
  pages={1-8}
}
Statistical analysis is widely used for countless scientific applications in order to analyze and infer meaning from data. A key challenge of any statistical analysis package aimed at large-scale, distributed data is to address the orthogonal issues of parallel scalability and numerical stability. In this paper we derive a series of formulas that allow for single-pass, yet numerically robust, pairwise parallel and incremental updates of both arbitrary-order centered statistical moments and co… Expand
Computing Contingency Statistics in Parallel: Design Trade-Offs and Limiting Cases
Design and Performance of a Scalable, Parallel Statistics Toolkit
Distributed, Numerically Stable Distance and Covariance Computation with MPI for Extremely Large Datasets
  • D. Peralta, Y. Saeys
  • Computer Science
  • 2019 IEEE International Congress on Big Data (BigDataCongress)
  • 2019
Feature-Based Statistical Analysis of Combustion Simulation Data
Interactive Exploration and Analysis of Large-Scale Simulations Using Topology-Based Data Segmentation
Large Scale Data Analysis
...
1
2
3
4
5
...

References

SHOWING 1-10 OF 13 REFERENCES
Map-Reduce for Machine Learning on Multicore
Note on a Method for Calculating Corrected Sums of Squares and Products
Updating formulae and a pairwise algorithm for computing sample variances
Performance of chromatic dispersion monitoring using statistical moments of asynchronously sampled waveform histograms
TORQUE resource manager
Computing higher-order moments online
  • Computing higher-order moments online
  • 2008
Dns of a turbulent lifted ethylene/air jet flame in an auto-ignitive coflow -stabilization and flame structure
  • Turbulent Non-premixed Flames Workshop (TNF'08)
  • 2008
...
1
2
...