Experimental performance evaluation of batch means procedures for simulation output analysis


We summarize the results of an extensive experimental performance evaluation of selected batch means procedures for building a confidence interval for a steady-state expected simulation response. We compare the performance of the well-known ABATCH and LBATCH procedures versus ASAP, a recently proposed variant of the method of nonoverlapping batch means (NOBM) that operates as follows: the batch size is progressively increased until either (a) the batch means pass the von Neumann test for independence, and then ASAP delivers a classical NOBM confidence interval; or (b) the batch means pass the Shapiro-Wilk test for multivariate normality, and then ASAP delivers a correlation-adjusted confidence interval. The latter correction is based on an inverted Cornish-Fisher expansion for the classical NOBM <i>t</i>-ratio, where the terms of the expansion are estimated via an autoregressive-moving average time series model of the batch means. Applying ABATCH, ASAP, and LBATCH to the analysis of a suite of twenty test problems involving discrete-time Markov chains, time-series processes, and queueing systems, we found ASAP to deliver confidence intervals that not only satisfy a user-specified absolute or relative precision requirement but also frequently outperform the corresponding confidence intervals delivered by ABATCH and LBATCH with respect to coverage probability.

Extracted Key Phrases

13 Figures and Tables

Cite this paper

@inproceedings{Steiger2000ExperimentalPE, title={Experimental performance evaluation of batch means procedures for simulation output analysis}, author={Natalie M. Steiger and James R. Wilson}, booktitle={Winter Simulation Conference}, year={2000} }