War of the benchmark means: time for a truce

@article{Mashey2004WarOT,
  title={War of the benchmark means: time for a truce},
  author={John R. Mashey},
  journal={SIGARCH Computer Architecture News},
  year={2004},
  volume={32},
  pages={1-14}
}
For decades, computer benchmarkers have fought a War of Means. Although many have raised concerns with the geometric mean (GM), it continues to be used by SPEC and others. This war is an unnecessarymisunderstanding due to inadequately articulated implicit assumptions, plus confusio namong populations, their parameters, sampling methods, and sample statistics. In fact, all the Means have their uses, sometimes in combination. Metrics may be algebraically correct, but statistically irrelevant or… CONTINUE READING
Highly Cited
This paper has 34 citations. REVIEW CITATIONS