1.3: The Process of Validating a Future State with Models

The simulation process used throughout this book is presented in this section.

Using simulation in designing or improving a system is itself a process. We summarize these steps into five strategic process phases (Standridge and Brown-Standridge, 1994; Standridge, 1998), which are similar to those in Banks, Carson, Nelson, and Nicol (2009). The strategic phases and the tactics used in each are shown in Table 1-1.

The first strategic phase in the simulation project process is the definition of the system design or improvement issues to be resolved and the characteristics of a solution to these issues. This requires identification of the system objects and system outputs that are relevant to the problem as well as the users of the system outputs and their requirements. Alternatives thought to result in system improvement are usually proposed. The scope of the model is defined, including the specification of which elements of a system are included and which are excluded. The quantities used to measure system performance are defined. All assumptions on which the model is based are stated. All of the above items should be recorded in a document. The contents of such a document is often referred to as the conceptual model. A team of simulation analysts, system experts, and managers performs this phase.

The construction of models of the system under study is the focus of the second phase. Simulation models are constructed as described in the next chapter. If necessary to aid in the construction of the simulation model, descriptive models such as flowcharts may be built.

Gaining an understanding of a system requires gathering and studying data from the system if it exists or the design of the system if it is proposed. Simulation model parameters are estimated using system data.

The simulation model is implemented as a computer program. Simulation software environments include model builders that provide the functionality and a user interface tailored to model building as well as automatically and transparently preparing the model for simulation.

Table 1-1: Phases and Tactics of the Simulation Project Process
Strategic Phase	Tactics
1. Define the Issues and Solution Objective	1. Identify the system outputs as well as the people who use them and their requirements. 2. Identify the systems that produce the outputs and individuals responsible for these systems. 3. Propose initial system alternatives that offer solution possibilities. 4. Identify all elements of the system that are to be included in the model. 5. State all modeling assumptions. 6. Specify system performance measures.
2. Build Models	1. Construct and implement simulation models. 2. Acquire data from the system or its design and estimate model parameters.
3. Identify Root Causes and Assess Initial Alternatives	1. Verify the model 2. Validate the model. 3. Design and perform the simulation experiments. 4. Analyze and draw inferences from the simulation results about system design and improvement issues. 5. Examine previously collected system data to aid in inference drawing.
4. Review and Extend Previous Work	1. Meet with system experts and managers. 2. Modify the simulation model and experiment. 3. Make additional simulation runs, analyze the results and draw inferences.
5. Implement Solutions and Evaluate	1. Modify system designs or operations. 2. Monitor system performance.

The third strategic phase involves identifying the system operating parameters, control strategies, and organizational structure that impact the issues and solution objectives identified in the first phase. Cause and effect relationships between system components are uncovered. The most basic and fundamental factors affecting the issues of interest, the root causes, are discovered. Possible solutions proposed during the first phases are tested using simulation experiments. Verification and validation are discussed in the next section as well as in Chapter 3.

Information resulting from experimentation with the simulation model is essential to the understanding of a system. Simulation experiments can be designed using standard design of experiments methods. At the same time, as many simulation experiments can be conducted as computer time and the limits on project duration allows. Thus, experiments can be replicated as needed for greater statistical precision, designed sequentially by basing future experiments on the results of preceding ones, and repeated to gather additional information needed for decision making.

The fourth strategic phase begins with a review of the work accomplished in phases one through three. This review is performed by a team of simulation analysts, system experts, and managers. The results of these meetings are often requests for additional alternatives to be evaluated, additional information to be extracted from simulation experiments, more detailed models of system alternatives, and even changes in system issues and solution objectives. The extensions and embellishments defined in this phase are based on conclusions drawn from the system data, simulation model, and simulation experiment results. The fourth stage relies on the ability to adapt simulation models during the course of a project and to design simulation experiments sequentially. Alternative solutions may be generated using formal ways for searching a solution space such as a response surface method. In addition, system experts may suggest alternative strategies, for example alternative part routings based on the work-in-process inventory at workstations. Performing additional experiments involves modifications to the simulation model as well as using new model parameter values.

Physical experiments using the actual system or laboratory prototypes of the system may be performed to confirm the benefits of the selected system improvements.

In the fifth phase, the selected improvements are implemented and the results monitored.

The iterative nature of the simulation project process should be emphasized. At every phase, new knowledge about the system and its behavior is gained. This may lead to a need to modify the work performed at any preceding phase. For example, the act of building a model, phase 2, may lead to a greater understanding of the interaction between system components as well as to redoing phase 1 to restate the solution objectives. Analyzing the simulation results in phase 3 may point out the need for more detailed information about the system. This can lead to the inclusion of additional system components in the model as phase 2 is redone.

Sargent (2009) states that model credibility has to do with creating the confidence managers and systems experts require in order to use a model and the information derived from that model for decision making. Credibility should be created as part of the simulation process. Managers and systems experts are included in the development of the conceptual model in the first strategic phase. They review the results of the second and third phases including model verification and validation as well as suggesting model modifications and additional experimentation. Simulation results must include quantities of interest to managers and systems experts as well as being reported in a format that they are able to review independently. Simulation input values should be organized in a way, such as a spreadsheet, that they understand and can review. Thus, managers and systems experts are an integral part of a simulation project and develop a sense of ownership in it.

Performing the first and last steps in the improvement process requires knowledge of the context in which the system operates as well as considerable time, effort, and experience. In this book, the first step will be given as part of the statement of the application studies and exercises and the last step assumed to be successful. Emphasis is given to building models, conducting experiments, and using the results to help validate and improve the future state of a transformed or new system.