This Article Full Text (PDF) References Alert me when this article is cited Alert me if a correction is posted Services Email this article to a friend Similar articles in this journal Alert me to new issues of the journal Download to citation manager Citing Articles Citing Articles via HighWire Citing Articles via Google Scholar Google Scholar Articles by Boesel, J. Articles by Kim, S.-H. Search for Related Content

Using Ranking and Selection to "Clean Up" after Simulation Optimization

The MITRE Corporation, 1820 Dolley Madison Boulevard, McLean, Virginia 22102
Department of Industrial Engineering and Management Sciences, Northwestern University, Evanston, Illinois 60208-3119
School of Industrial and Systems Engineering, Georgia Institute of Technology, Atlanta, Georgia 30332
boesel{at}mitre.org
nelsonb{at}northwestern.edu
skim{at}isye.gatech.edu

In this paper we address the problem of finding the simulated system with the best (maximum or minimum) expected performance when the number of systems is large and initial samples from each system have already been taken. This problem may be encountered when a heuristic search procedure—perhaps one originally designed for use in a deterministic environment—has been applied in a simulation-optimization context. Because of stochastic variation, the system with the best sample mean at the end of the search procedure may not coincide with the true best system encountered during the search. This paper develops statistical procedures that return the best system encountered by the search (or one near the best) with a prespecified probability. We approach this problem using combinations of statistical subset selection and indifference-zone ranking procedures. The subset-selection procedures, which use only the data already collected, screen out the obviously inferior systems, while the indifference-zone procedures, which require additional simulation effort, distinguish the best from the less obviously inferior systems.

Subject classifications: Simulation, statistical analysis: selecting the best system; Simulation, efficiency: large-scale screening; Programming/stochastic: terminal inference.
History: Received August 1999; revision received June 2002; accepted September 2002.

This article has been cited by other articles:

				J. Branke, S. E. Chick, and C. Schmidt Selecting a Selection Procedure Management Science, December 1, 2007; 53(12): 1916 - 1932. [Abstract] [PDF]

				V. Lesnevski, B. L. Nelson, and J. Staum Simulation of Coherent Risk Measures Based on Generalized Scenarios Management Science, November 1, 2007; 53(11): 1756 - 1769. [Abstract] [PDF]

				L. J. Hong and B. L. Nelson Discrete Optimization via Simulation Using COMPASS Operations Research, January 1, 2006; 54(1): 115 - 129. [Abstract] [PDF]