Current design decisions must be made while considering uncertainty in both models of the design and inputs to the design. In most cases, high fidelity models are used with the assumption that the resulting model uncertainties are insignificant to the decision making process. This paper presents a methodology for managing uncertainty during system-level conceptual design of complex multidisciplinary systems. This methodology is based upon quantifying the information available in a set of observations of computationally expensive subsystem models with more computationally efficient kriging models. By using kriging models, the computational expense of a Monte Carlo simulation to assess the impact of the sources of uncertainty on system-level performance parameters becomes tractable. The use of a kriging model as an approximation to an original computer model introduces model uncertainty, which is included as part of the methodology. The methodology is demonstrated as a decision-making tool for the design of a satellite system.

1.
Hazelrigg
,
G. A.
, 1996,
Systems Engineering: An Approach to Information-Based Design
,
Prentice Hall International Series in Industrial and Systems Engineering
,
Prentice Hall
,
Upper Saddle River, NJ
.
2.
Maddulapalli
,
K.
,
Azarm
,
S.
, and
Boyars
,
A.
, 2005, “
Interactive Product Design Selection With an Implicit Value Function
,”
ASME J. Mech. Des.
1050-0472,
127
, pp.
367
377
.
3.
Lewis
,
K.
, and
Mistree
,
F.
, 1998, “
The Other Side of Multidisciplinary Design Optimization: Accomodating a Multiobjective, Uncertain and Non-Deterministic World
,”
Eng. Optimiz.
0305-215X,
31
, pp.
161
189
.
4.
Chen
,
V. C. P.
,
Tsui
,
K.-L.
,
Barton
,
R. R.
, and
Allen
,
J. K.
, 2003, “
A Review of Design and Modeling in Computer Experiments
,”
Handbook of Statistics
;
R.
Khattree
and
C.
Rao
, eds.,
Elsevier Science
,
Amsterdam
, Vol.
22
, pp.
231
261
.
5.
Simpson
,
T. W.
,
Booker
,
A. J.
,
Ghosh
,
S.
,
Giunta
,
A.
,
Koch
,
P. N.
, and
Yang
,
R. J.
, 2004, “
Approximation Methods in Multidisciplinary Analysis and Optimization: A Panel Discussion
,”
Struct. Multidiscip. Optim.
1615-147X,
27
(
5
), pp.
302
313
.
6.
Cox
,
D. D.
, and
John
,
S.
, 1997, “
SDO: A Statistical Method for Global Optimization
,”
Multidisciplinary Design Optimization: State of the Art
,
N. M.
Alexandrov
and
M. Y.
Hussaini
, eds.,
SIAM
,
Philadelphia
, pp.
315
329
.
7.
Trosset
,
M. W.
, and
Torczon
,
V.
, 1997, “
Numerical Optimization Using Computer Experiments
,” ICASE Report No. 97-38,
NASA Langley Research Center
, Hampton, VA.
8.
Booker
,
A. J.
,
Conn
,
A. R.
,
Dennis
, Jr.,
J. E.
Frank
,
P. D.
,
Trosset
,
M.
, and
Torczon
,
V.
, 1995, “
Global Modeling for Optimization: Boeing/IBM/Rice Collaborative Project 1995 Final Report
,” ISSTECH-95–032,
The Boeing Company
, Bellevue, WA.
9.
Sasena
,
M. J.
,
Papalambros
,
P.
, and
Goovaerts
,
P.
, 2002, “
Exploration of Metamodeling Sampling Criteria for Constrained Global Optimization
,”
Eng. Optimiz.
0305-215X,
34
(
3
), pp.
263
278
.
10.
Osio
,
I. G.
, and
Amon
,
C. H.
, 1996, “
An Engineering Design Methodology With Multistage Bayesian Surrogate and Optimal Sampling
,”
Res. Eng. Des.
0934-9839,
8
(
4
), pp.
189
206
.
11.
Simpson
,
T. W.
,
Maurey
,
T. M.
,
Korte
,
J. J.
, and
Mistree
,
F.
, 2001, “
Kriging Metamodels for Global Approximation in Simulation-Based Multidisciplinary Design Optimization
,”
AIAA J.
0001-1452,
39
(
12
), pp.
2233
2241
.
12.
Goovaerts
,
P.
, 1997,
Geostatistics for Natural Resources Evaluation
,
Applied Geostatistics Series
,
Oxford University Press
,
New York
.
13.
Gotway
,
C. A.
, 1994, “
The Use of Conditional Simulation in Nuclear-Waste-Site Performance Assessment
,”
Technometrics
0040-1706,
36
(
2
), pp.
129
141
.
14.
Martin
,
J. D.
, and
Simpson
,
T. W.
, 2005, “
On the Use of Kriging Models to Approximate Deterministic Computer Models
,”
AIAA J.
0001-1452,
43
(
4
), pp.
853
863
.
15.
Bayarri
,
M. J.
,
Berger
,
J. O.
,
Higdon
,
D.
,
Kennedy
,
M. C.
,
Kottas
,
A.
,
Paulo
,
R.
,
Sacks
,
J.
,
Cafeo
,
J. A.
,
Cavendish
,
J. C.
,
Lin
,
C. H.
, and
Tui
,
J.
, 2002, “
A Framework for Validation of Computer Models
,” Tech. Rep. No. 128,
National Institute of Statistical Sciences
, Gaithersburg, MD.
16.
Den Hertog
,
D.
,
Kleijnen
,
J.
, and
Siem
,
A.
, 2005, “
The Correct Kriging Variance Estimated by Bootstrapping
,”
J. Oper. Res. Soc.
0160-5682,
57
(
4
), pp.
400
409
.
17.
Martin
,
J. D.
, and
Simpson
,
T. W.
, 2004, “
A Monte Carlo Simulation of the Kriging Model
,”
10th AIAA/ISSMO Symposium on Multidisciplinary Analysis and Optimization
, AIAA-2004–4483.
18.
Robert
,
C. P.
, and
Casella
,
G.
, 1999,
Monte Carlo Statistical Methods
,
Springer Texts in Statistics
,
Springer-Verlag
,
New York
.
19.
Huang
,
B.
, and
Du
,
X.
, 2006, “
Uncertianty Analysis by Dimension Reduction Integration and Saddlepoint Approximations
,”
ASME J. Mech. Des.
1050-0472,
128
, pp.
26
33
.
20.
Saltelli
,
A.
,
Tarantola
,
S.
, and
Chan
,
K.
, 1999, “
A Quantitative Model-Independent Method for Global Sensitivity Analysis of Model Ouptut
,”
Technometrics
0040-1706,
41
(
1
), pp.
39
56
.
21.
Martin
,
J. D.
, 2005, “
A Methodology for Evaluating System-level Uncertainty in the Conceptual Design of Complex Multidisciplinary Systems
,” Ph.D. Dissertation, Dept. of Mechanical Engineering, The Pennsylvania State University, University Park, PA.
22.
Saltelli
,
A.
,
Chan
,
K.
, and
Scott
,
E. M.
, 2000,
Mathematical and Statistical Methods: Sensitivity Analysis
,
Wiley Series in Probability and Statistics
,
Wiley
,
Chichester
.
23.
Chen
,
W.
,
Jin
,
R.
, and
Sudjianto
,
A.
, 2005, “
Analytical Variance-Based Global Sensitivity Analysis in Simulation-Based Design Under Uncertainty
,”
ASME J. Mech. Des.
1050-0472,
127
, pp.
875
886
.
24.
Charania
,
C. A.
,
Bradford
,
J. E.
,
Olds
,
J. R.
, and
Graham
,
M.
, 2002, “
System Level Uncertainty Assessment for Collaborative RLV Design
,”
JANNAF 38th Combustion Subcommitte/26th Airbreathing Propulsion Subcommittee/20th Propulsion Systems Hazards Subcommittee/2nd Modeling and Simulation Subcommittee Joint Meeting
, JANNAF-2002–2B-4-MSS.
25.
Qu
,
X.
, and
Haftka
,
R. T.
, 2003, “
Design Under Uncertainty Using Monte Carlo Simulation and Probabilistic Sufficiency Factor
,”
ASME 2003 Design Engineering Technical Conference and Computers and Information in Engineering Conference
, DETC2003/DAC-48704.
26.
Radhakrishnan
,
R.
, and
McAdams
,
D. A.
, 2005, “
A Methodology for Model Selection in Engineering Design
,”
ASME J. Mech. Des.
1050-0472,
127
, pp.
378
387
.
27.
Cooper
,
A. B.
,
Georgiopoulos
,
P.
,
Kim
,
H. M.
, and
Papalambros
,
P. Y.
, 2006, “
Analytical Target Setting: An Enterprise Context in Optimal Product Design
,”
ASME J. Mech. Des.
1050-0472,
128
, pp.
4
13
.
28.
McKay
,
M.
,
Conover
,
W.
, and
Beckman
,
R.
, 1979, “
A Comparison of Three Methods for Selecting Values of Input in the Analysis of Output from a Computer Code
,”
Technometrics
0040-1706,
21
(
1
), pp.
239
245
.
29.
Simpson
,
T. W.
,
Peplinski
,
J.
,
Koch
,
P. N.
, and
Allen
,
J. K.
, 2001, “
Metamodels for Computer-Based Engineering Design: Survey and Recommendations
,”
Eng. Comput.
0177-0667,
17
(
2
), pp.
129
150
.
30.
Simpson
,
T. W.
,
Lin
,
K. J.
, and
Chen
,
W.
, 2001, “
Sampling Stategies for Computer Experiments: Design and Analysis
,”
Int. J. Reliab. Appl.
1598-0073,
2
(
3
), pp.
209
240
.
31.
Martin
,
J. D.
, and
Simpson
,
T. W.
, 2005, “
On Using Kriging Models as Probabilistic Models in Design
,”
SAE Trans.
0096-736X,
5
, pp.
129
139
.
32.
Martin
,
J. D.
, and
Simpson
,
T. W.
, 2005, “
On the Use of Kriging Models to Approximate Deterministic Computer Models
,”
AIAA J.
0001-1452,
43
(
4
), pp.
853
863
.
33.
Myers
,
R. H.
, and
Montgomery
,
D. C.
, 2002,
Response Surface Methodology: Process and Product Optimization Using Designed Experiments
,
Wiley Series in Probability and Statistics
,
2nd ed.
,
Wiley
,
New York
.
34.
Burnham
,
K. P.
, and
Anderson
,
D. R.
, 2002,
Model Selection and Multimodel Inference: A Practical Information-Theoretic Approach
,
2nd ed.
,
Springer-Verlag
,
New York
.
35.
Du
,
X.
,
Sudjianto
,
A.
, and
Huang
,
B.
, 2005, “
Reliability-Based Design with the Mixture of Random and Interval Variables
,”
ASME J. Mech. Des.
1050-0472,
127
, pp.
1068
1076
.
36.
Xiong
,
Y.
, and
Rao
,
S. S.
, 2005, “
A Fuzzy Dynamic Programming Approach for the Mixed-Discrete Optimziation of Mechanical Systems
,”
ASME J. Mech. Des.
1050-0472,
127
, pp.
1088
1099
.
37.
Helton
,
J. C.
, and
Davis
,
F. J.
, 2000, “
Sampling-Based Methods
,”
Sensitivity Analysis
,
A.
Saltelli
,
K.
Chan
, and
E. M.
Scott
, eds.,
Wiley
,
Chichester
, pp.
101
153
.
38.
Mawardi
,
A.
, and
Pitchumani
,
R.
, 2005, “
SAMS: Stochastic Analysis With Minimal Sampling—A Fast Algorithm for Analysis and Design Under Uncertainty
,”
ASME J. Mech. Des.
1050-0472,
127
, pp.
558
571
.
39.
Yukish
,
M.
, 2004, “
Algorithms to Identify Pareto Points in Multi-Dimensional Data Sets
,” Ph.D. Dissertation, Dept. of Mechanical Engineering, The Pennsylvania State University, University Park, PA.
40.
McKay
,
M.
, 1995, “
Evaluating Prediction Uncertainty
,” NUREG/CR-6311, Los Alamos National Laboratory.
41.
Abell
,
M. L.
,
Braselton
,
J. P.
, and
Rafter
,
J. A.
, 1999,
Statistics with Mathematica
,
Academic Press
,
San Diego
.
42.
Booker
,
A. J.
, 2000, “
Well-Conditioned Kriging Models for Optimization of Computer Simulations
,”
M&CT-TECH-00–002
,
The Boeing Company
,
Bellevue, WA
.
43.
Kleijnen
,
J. P. C.
, and
Van Beers
,
W. C. M.
, 2004, “
Application-Driven Sequential Designs for Simulation Experiments: Kriging Metamodeling
,”
J. Oper. Res. Soc.
0160-5682,
55
(
8
), pp.
876
883
.
44.
Spall
,
J. C.
, 2003,
Introduction to Stochastic Search and Optimization; Estimation, Simulation, and Control
,
Wiley
,
Hoboken, NJ
.
45.
Lophaven
,
S. N.
,
Nielsen
,
H. B.
, and
Sondergaard
,
J.
, 2002, “
DACE: A MATLAB Kriging Toolbox, Version 2.0
,” IMM Technical Unversity of Denmark, Lyngby.
You do not currently have access to this content.