Computational simulation models support a rapid design process. Given model approximation and operating conditions uncertainty, designers must have confidence that the designs obtained using simulations will perform as expected. The traditional approach to address this need consists of model validation efforts conducted predominantly prior to the optimization process. We argue that model validation is too daunting of a task to be conducted with meaningful success for design optimization problems associated with high-dimensional space and parameter spaces. In contrast, we propose a methodology for maximizing confidence in designs generated during the simulation-based optimization process. Specifically, we adopt a trust-region-like sequential optimization process and utilize a Bayesian hypothesis testing technique to quantify model confidence, which we maximize by calibrating the simulation model within local domains if and when necessary. This ensures that the design iterates generated during the sequential optimization process are associated with maximized confidence in the utilized simulation model. The proposed methodology is illustrated using a cantilever beam design subject to vibration.

References

References
1.
Bayarri
,
M. J.
,
Berger
,
J. O.
,
Paulo
,
R.
,
Sacks
,
J.
,
Cafeo
,
J. A.
,
Cavendish
,
J.
,
Lin
,
C. H.
, and
Tu
,
J.
,
2007
, “
A Framework for Validation of Computer Models
,”
Technometrics
,
49
(
2
), pp.
138
154
.10.1198/004017007000000092
2.
Kennedy
,
M. C.
, and
O'Hagan
,
A.
,
2001
, “
Bayesian Calibration of Computer Models
,”
J. R. Stat. Soc., Ser. B
,
63
, pp.
425
450
.10.1111/1467-9868.00294
3.
Booker
,
A. J.
,
Dennis
, Jr.,
J. E.
,
Frank
,
P. D.
,
Serafini
,
D. B.
,
Torczon
,
V.
, and
Trosset
,
M. W.
,
1999
, “
A Rigorous Framework by Surrogates for Optimization of Expensive Functions
,”
Struct. Optim.
,
17
, pp.
1
13
.10.1007/BF01197708
4.
DoD Directive No. 5000.61
, “
Modeling and Simulation (M&S), Verification, validation, and Accreditation (VV&A)
,”
Defense Modeling and Simulation Office
, www.dmso.mil/docslib.
5.
Oberkampf, W. L.
, and
Barone, M. F.
,
2006
, “Measures of Agreement between Computation and Experiment: Validation Metrics,” Journal of Computational Physics,
217
(1), pp. 5–36.
6.
Gu
,
L.
, and
Yang
,
R. J.
,
2003
, “
Recent Applications on Reliability-Based Optimization of Automotive Structures
,”
SAE World Congress
,
Detroit, MI
, Paper No. 2003-01-0152.
7.
Oden
,
J. T.
,
2006
, Chair, “
Revolutionizing Engineering Science Through Simulation: The NSF Blue Ribbon Panel on Simulation-Based Engineering Science
,”
National Science Foundation
.
8.
Oberkampf
,
W. L.
,
Trucano
,
T. G.
, and
Hirsch
,
C.
,
2004
, “
Verification, Validation, and Predictive Capability in Computational Engineering and Physics
,”
Appl. Mech. Rev.
,
57
(
5
), pp.
345
384
.10.1115/1.1767847
9.
Easterling, R. G. and Berger, J. O.,
2002
, “Statistical Foundations for the Validation of Computer Models,”
Presentation at Computer Model Verification and Validation in the 21st Century Workshop
, Johns Hopkins University, MD.
10.
Zhang
,
R.
and
Mahadevan
,
S.
,
2003
, “
Bayesian Methodology for Reliability Model Acceptance
,”
Reliab. Eng. Syst. Saf.
,
80
, pp.
95
103
.10.1016/S0951-8320(02)00269-7
11.
Mahadevan
,
S.
, and
Rebba
,
R.
,
2005
, “
Validation of Reliability Computational Models using Bayes Networks
,”
Reliab. Eng. Syst. Saf.
,
87
(
2
), pp.
223
232
.10.1016/j.ress.2004.05.001
12.
Rebba
,
R.
, and
Mahadevan
,
S.
,
2006
, “
Model Predictive Capability Assessment Under Uncertainty
,”
AIAA J.
,
44
(
10
), pp.
2376
2384
.10.2514/1.19103
13.
Jiang
,
X.
, and
Mahadevan
,
S.
,
2008a
, “
Bayesian Validation Assessment of Multivariate Computational Models
,”
J. Appl. Stat.
,
35
(
1
), pp.
49
65
.10.1080/02664760701683577
14.
Jiang
,
X.
, and
Mahadevan
,
S.
,
2008b
, “
Bayesian Wavelet Method for Multivariate Model Assessment of Dynamical Systems
,”
J. Sound Vib.
,
312
(
4–5
), pp.
694
712
.10.1016/j.jsv.2007.11.025
15.
Jiang
,
X.
, and
Mahadevan
,
S.
,
2009
, “
Bayesian Inference Method for Model Validation and Confidence Extrapolation
,”
J. Appl. Stat.
,
36
(
6
), pp.
659
677
.10.1080/02664760802499295
16.
Gunawan
,
S.
and
Papalambros
,
P. Y.
,
2006
, “
A Bayesian Approach to Reliability-Based Optimization With Incomplete Information
,”
ASME J. Mech. Des.
,
128
(
4
), pp.
909
918
.10.1115/1.2204969
17.
Chen
,
W.
,
Xiong
,
Y.
,
Tsui
,
K.-L.
, and
Wang
,
S.
,
2008
, “
A Design-Driven Validation Approach Using Bayesian Prediction Models
,”
ASME J. Mech. Des.
,
130
(
2
), p.
021101
.10.1115/1.2809439
18.
Easterling
,
R. G.
,
2003
, “
Statistical Foundations for Model Validation
,”
Sandia National Laboratories, Albuquerque, NM.
Technical Report No. SAND2003-0287.
19.
Oberkampf
,
W. L.
,
Trucano
,
T. G.
, and
Hirsch
,
C.
,
2003
, “
Verification, Validation, and Predictive Capabilities in Computational Engineering and Physics
,”
Sandia National Laboratories
, Technical Report Sand. No 2003-3769, Albuquerque, NM.
20.
Rebba
,
R.
,
2005
,
Model Validation and Design under Uncertainty
, Ph.D. thesis,
Vanderbilt University
, Nashville, TN.
21.
Hemez
,
F. M.
, and
Doebling
,
S. W.
,
2000
, “
Validation of Structural Dynamics Models at Los Alamos National Laboratory
,”
Proceedings, 41st AIAA/ASME/ASCE/AHS/ASC Structures, Structural Dynamics and Materials Conference
,
Atlanta, GA
.
22.
Jiang
,
X.
, and
Mahadevan
,
S.
,
2007
, “
Bayesian Risk-based Decision Method for Model Validation Under Uncertainty
,”
Reliab. Eng. Syst. Saf.
,
92
(
6
), pp.
707
718
.10.1016/j.ress.2006.03.006
23.
Chen
,
W.
,
Baghdasaryan
,
L.
,
Buranathiti
,
T.
, and
Cao
,
J.
,
2004
, “
Model Validation via Uncertainty Propagation and Data Transformations
,”
AIAA J.
,
42
(
7
), pp.
1406
1415
.10.2514/1.491
24.
Jiang
,
X.
,
Yang
,
R.-J.
,
Barbat
,
S.
, and
Weerappuli
,
P.
,
2009
, “
Bayesian Probabilistic PCA Approach for Model Validation of Dynamic Systems
,”
SAE Int. J. Mate. Manuf.
,
2
(
1
), pp.
555
563
.10.4271/2009-01-1404
25.
Tipping
,
M. E.
, and
Bishop
,
C. M.
,
1999
, “
Probabilistic Principal Component Analysis
,”
J. R. Stat. Soc. Ser. B (Stat. Methodol.)
,
61
(
3
), pp.
611
622
.10.1111/1467-9868.00196
26.
Joliffe
,
2002
, I. T.,
Principal Component Analysis
,
Springer
,
New York
.
27.
Pai
,
Y.
,
2009
,
Investigation of Bayesian Model Validation Framework for Dynamic Systems
, M.S. thesis,
Department of Mechanical Engineering, University of Michigan, Ann Arbor
.
28.
Gelman
,
A.
,
Carlin
,
J. B.
,
Stern
,
H. S.
, and
Rubin
,
D. B.
,
2003
,
Bayesian Data Analysis
,
Taylor & Francis, Inc.
, pp.
85
86
.
29.
Rao
,
S. S.
,
2004
,
Mechanical Vibrations
,
4th ed.
,
Pearson Prentice–Hall
, Upper Saddle River, NJ.
30.
Drignei
,
D.
,
Mourelatos
,
Z. P.
, and
Rebba
,
R.
,
2010
, “
Parameter Screening in Dynamic Computer Model Calibration Using Global Sensitivities
,”
Proceedings, 2010 ASME IDETC/CIE
,
Montreal, Canada
, Paper No. DETC2010-28343.
You do not currently have access to this content.