4.4: Common Design Elements

The elements common to all simulation experiments are discussed in the following sections. These include model parameters and their values, performance measures, and random number streams.

4.4.1 Model Parameters and Their Values

Model parameters correspond to system control variables or operational rules whose values can be changed to meet the solution objectives defined in the first step of the simulation process. Values of model parameters can be quantitative such as the size of a buffer or qualitative such as which routing policy to use.

Often in traditional experimental design and analysis, time and cost constraints result in the use of only two or three values of each model parameter. Simulation affords the opportunity to test as many values as time and computing resources allow. For example, various sizes of an inter- station buffer could be simulated. A very large size could represent an infinite buffer. A buffer size of one or two would be minimal. Intermediate buffer sizes such as five and ten could be evaluated.

Which values are used may depend on the results of preceding simulations. For example, results of the initial simulations may indicate that a buffer size in the range 10 to 20 is needed. Additional simulations would be run with for buffer sizes between 10 and 20.

Model parameters must be defined and their values specified.

4.4.2 Performance Measures

Performance measures are quantities used to evaluate system behavior. They are defined in accordance with principle 9 of chapter 1: “Simulation experimental results conform to unique system requirements for information.” Thus, each simulation experiment could have different performance measures.

Possible performance measures for experiments with the two stations in a series model could be as follows:

1. The number of items waiting in each buffer.
2. The percentage of time each workstation is busy.
3. The percentage of time each workstation is idle.
4. The time an item spends in the system (lead time).
5. The total number of items processed by the workstation.

Note that state variable values are used as performance measures along with the time taken by entities in one, more than one, or all of the processing steps. A count of the number of entities completing processing is desired as well. These kinds of performance measures are typical of many simulation experiments.

Performance measures must be defined, including how each is computed.

4.4.3 Streams of Random Samples

One purpose of a simulation experiment is comparing scenarios. Suppose that no statistically significant difference between two scenarios is found. This could occur because the scenarios do not cause distinct differences in system performance. A second and undesirable possibility is that the variance of the observations made during the simulation is too high to permit true differences in system observations to be confirmed statistically.

Suppose we wished to assess a change in the operation of workstation A in the two stations in a series model where the range of the operation time is reduced to uniformly distributed between 7 and 11 seconds from uniformly distributed between 5 and 13 seconds. The same arrivals could be used in simulating both scenarios. Thus, the comparison could be made with respect to the same set of entities processed by the workstation. In general, this approach is referred to as the method of common random numbers since simulations of the two scenarios have the same pattern of arrivals in common. Each time between arrivals was determined by taking a random sample from the exponential distribution modeling this quantity. How this is done will be discussed in the next chapter.

To better understand the effect of common random numbers, consider what happens when they are not used. There would be a different set of arrivals in the simulation of the first scenario than in the simulation of the second scenario. Observed differences in performance measure values between the two scenarios could be due to the differences in the arrivals or true differences between the scenarios. Thus, the variance associated with summary statistics of differences in values, such as the mean lead time, would likely be higher than if common random numbers were used. This higher variance might result in a failure to detect a true difference between the scenarios with respect to a given performance measure such as lead time even if such a difference existed.

The method of common random numbers requires distinct streams of samples for each quantity modeled by a probability distribution. While this does not guarantee a reduction in the variance of the difference, experience has shown that a reduction often occurs. In practice for most simulation languages, this means that the stream of samples associated with each quantity modeled by a probability distribution must be given a distinct name.

Law (2007) more details concerning the common random number approach as well as other experiment design techniques to control the variance. Banks, Carson, Nelson, and Nicol (2009) discuss these techniques as well.

The quantities modeled by probability distributions in a model must be identified and uniquely named the method of common random numbers may be employed.