Design of experiments: robust design: seeking the best of all possible worlds

ABSTRACT

We describe a framework for analyzing simulation output in order to find solutions that will work well after implementation. We show how the use of a loss function that incorporates both system mean and system variability can be used to efficiently and effectively carry out system optimization and improvement efforts. For models whose behavior depends on quantitative factors, we illustrate how robust design can be accomplished by using simple experimental designs in conjunction with response-surface metamodels. The results can yield new insights into system behavior, and may lead to recommended system configurations that differ substantially from those selected by analysis solely on the basis of mean response. We assume a knowledge base at the level of Chapter 12 of Simulation Modeling and Analysis (Law and Kelton 2000) but will review essential elements and distribute illustrative examples at the session.

REFERENCES

• 1: George Box , Anne C. Shoemaker , Kwok-Leung Tsui , Ramon V. León , William C. Parr , V. N. Nair , D. Pregibon , Raymond J. Carroll , David Ruppert , Berton Gunter , Neil R. Ullman, Signal-to-noise ratios, performance criteria, and transformations, Technometrics, v.30 n.1, p.1-40, Feb. 1988 [doi>10.2307/1270311]
• 2: George E P Box , Norman R Draper, Empirical model-building and response surface, John Wiley & Sons, Inc., New York, NY, 1986
• 3: Box, G. E. P., W. G. Hunter and J. S. Hunter. 1978. Statistics for experimenters. New York: John Wiley and Sons, Inc.
• 4: Joan M. Donohue , Ernest C. Houck , Raymond H. Myers, Simulation designs for quadratic response surface models in the presence of model misspecification, Management Science, v.38 n.12, p.1765-1791, Dec. 1992 [doi>10.1287/mnsc.38.12.1765]
• 5: Sarah J. Hood , Peter D. Welch, Response surface methodology and its application in simulation, Proceedings of the 25th conference on Winter simulation, p.115-122, December 12-15, 1993, Los Angeles, California, United States [doi>10.1145/256563.256598]
• 6: Sheldon H. Jacobson , Arnold H. Buss , Lee W. Schruben, Driving frequency selection for frequency domain simulation experiments, Operations Research, v.39 n.6, p.917-924, Nov.-Dec. 1991 [doi>10.1287/opre.39.6.917]
• 7: Averill M. Law , David M. Kelton, Simulation Modeling and Analysis, McGraw-Hill Higher Education, 1999
• 8: Mousumi Mitra , Stephen K. Park, Solution to the indexing problem of frequency domain simulation experiments, Proceedings of the 23rd conference on Winter simulation, p.907-915, December 08-11, 1991, Phoenix, Arizona, United States
• 9: Moeeni, F., S. M. Sanchez and A. J. Vakharia. 1997. A robust design methodology for kanban system design. International Journal of Production Research 35 (10): 2821-2838.
• 10: Montgomery, D. C. 1990. Design and analysis of experiments. New York: John Wiley and Sons, Inc.
• 11: Myers, R. H., A. I. Khuri and G. Vining. 1992. Response surface alternatives to the Taguchi robust parameter design approach. The American Statistician 46 (2): 131-139.
• 12: Vijayan N. Nair , Bovas Abraham , Jock MacKay , George Box , Raghu N. Kacker , Thomas J. Lorenzen , James M. Lucas , Raymond H. Myers , G. Geoffrey Vining , John A. Nelder , Madhav S. Phadke , Jerome Sacks , William J. Welch , Anne C. Shoemaker , Kwok L. Tsui , Shin Taguchi , C. F. Jeff Wu, Taguchi's parameter design: a panel discussion, Technometrics, v.34 n.2, p.127-161, May 1992 [doi>10.2307/1269231]
• 13: Pignatiello, J. J. Jr. and J. S. Ramberg. 1991. Top ten triumphs and tragedies of Genichi Taguchi. Quality Engineering 4 (2): 211-235.
• 14: Ramberg, J. S., J. J. Pignatiello, Jr. and S. M. Sanchez. 1992. A critique and enhancement of the Taguchi method. ASQC Quality Congress Transactions 491-498.
• 15: John S. Ramberg , Susan M. Sanchez , Paul J. Sanchez , Ludwig J. Hollick, Designing simulation experiments: Taguchi methods and response surface metamodels, Proceedings of the 23rd conference on Winter simulation, p.167-176, December 08-11, 1991, Phoenix, Arizona, United States
• 16: Sacks, J., W. J. Welch, T. J. Mitchell and H. P. Wynn. 1989. Design and analysis of computer experiments (with discussion). Statistical Science 4:409-435.
• 17: Paul J. Sanchez , Arnold H. Buss, A model for frequency domain experiments, Proceedings of the 19th conference on Winter simulation, p.424-427, December 14-16, 1987, Atlanta, Georgia, United States [doi>10.1145/318371.318628]
• 18: Paul J. Sanchez , Susan M. Sanchez, Design of frequency-domain experiments for discrete-valued factors, Applied Mathematics and Computation, v.42 n.1, p.1-21, March 1991 [doi>10.1016/0096-3003(91)90111-Y]
• 19: Susan M. Sanchez, A robust design tutorial, Proceedings of the 26th conference on Winter simulation, p.106-113, December 11-14, 1994, Orlando, Florida, United States
• 20: Susan M. Sanchez, Experimental designs for system assessment and improvement when noise factors are correlated, Proceedings of the 26th conference on Winter simulation, p.290-296, December 11-14, 1994, Orlando, Florida, United States
• 21: Sanchez, S. M., J. S. Ramberg, J. Fiero and J. J. Pignatiello, Jr. 1993. Quality by design. Ch. 10 in Concurrent engineering: Automation, tools, and techniques, ed. A. Kusiak, 235-286. New York: John Wiley and Sons, Inc.
• 22: Sanchez, S. M., P. J. Sanchez and J. S. Ramberg. 1998. A simulation framework for robust system design. Ch. 12 in Concurrent Design of Products, Manufacturing Processes and Systems, ed. B. Wang, 279-314. New York: Gordon and Breach.
• 23: Susan M. Sanchez , L. Douglas Smith , Edward C. Lawrence, Sensitivity and scenario analysis for simulation metamodels, Proceedings of the 28th conference on Winter simulation, p.1440-1447, December 08-11, 1996, Coronado, California, United States [doi>10.1145/256562.256988]
• 24: Sanchez, S. M., L. D. Smith and E. C. Lawrence. 2000. Tolerance design revisited: Assessing the impact of correlated noise factors. Working paper, Operation Research Department, Naval Postgraduate School, Monterey, CA.
• 25: Lee W. Schruben , V. James Cogliano, An experimental procedure for simulation response surface model identification, Communications of the ACM, v.30 n.8, p.716-730, Aug. 1987 [doi>10.1145/27651.27656]
• 26: Schruben, L. W. and B. H. Margolin. 1978. Pseudo-random number assignment in statistically designed simulation and distribution sampling experiments. Journal of the American Statistical Association 73, 504-525.
• 27: Lee W. Schruben , Susan M. Sanchez , Paul J. Sanchez , Veronica A. Czitrom, Variance reallocation in Taguchi's robust design framework, Proceedings of the 24th conference on Winter simulation, p.548-556, December 13-16, 1992, Arlington, Virginia, United States [doi>10.1145/167293.167636]
• 28: Taguchi, G. 1986. Introduction to quality engineering. White Plains, New York: UNIPUB/Krauss International.
• 29: Taguchi, G. 1987. System of experimental design, vols. 1 and 2. White Plains, New York: UNIPUB/Krauss International.
• 30: Taguchi, G. and Y. Wu. 1980. Introduction to off-line quality control. Nagoya, Japan: Central Japan Quality Association.
• 31: Jeffrey D. Tew, Using central composite designs in simulation experiments, Proceedings of the 24th conference on Winter simulation, p.529-538, December 13-16, 1992, Arlington, Virginia, United States [doi>10.1145/167293.167634]
• 32: Jeffrey D. Tew , James R. Wilson, Validation of simulation analysis methods for the Schruben-Margolin correlation-induction strategy, Operations Research, v.40 n.1, p.87-103, Jan.–Feb. 1992 [doi>10.1287/opre.40.1.87]
• 33: Tew, J. D. and Wilson, J. R. 1994. Estimating simulation metamodels using combined correlation-based reduction techniques. IIE Transactions, 26 (3): 2-16.
• 34: Vining, G. G. and R. H. Myers. 1990. Combining Taguchi and response surface philosophies: A dual response approach. Journal of Quality Technology 22:38-45.
• 35: Welch, W. J., T. K. Yu, S. M. Kang, and J. Sacks. 1990. Computer experiments for quality control by robust design. Journal of Quality Technology 22:15-22.
• 36: Wild, R. H. and J. J. Pignatiello, Jr. 1991. An experimental design strategy for designing robust systems using discrete-event simulation. Simulation 57 (6): 358-368.