The optimal design of complex systems in engineering requires the availability of mathematical models of system’s behavior as a function of a set of design variables; such models allow the designer to search for the best solution to the design problem. However, system models (e.g., computational fluid dynamics (CFD) analysis, physical prototypes) are usually time-consuming and expensive to evaluate, and thus unsuited for systematic use during design. Approximate models of system behavior based on limited data, also known as metamodels, allow significant savings by reducing the resources devoted to modeling during the design process. In this work in engineering design based on multiple performance criteria, we propose the use of multi-response Bayesian surrogate models (MR-BSM) to model several aspects of system behavior jointly, instead of modeling each individually. To this end, we formulated a family of multiresponse correlation functions, suitable for prediction of several response variables that are observed simultaneously from the same computer simulation. Using a set of test functions with varying degrees of correlation, we compared the performance of MR-BSM against metamodels built individually for each response. Our results indicate that MR-BSM outperforms individual metamodels in 53% to 75% of the test cases, though the relative performance depends on the sample size, sampling scheme and the actual correlation among the observed response values. In addition, the relative performance of MR-BSM versus individual metamodels was contingent upon the ability to select an appropriate covariance/correlation function for each application, a task for which a modified version of Akaike’s Information Criterion was observed to be inadequate.

References

References
1.
Torczon
,
V.
, and
Trosset
,
M. W.
, 1998, “
Using Approximations to Accelerate Engineering Design Optimization
,” Institute for Computer Applications in Science and Engineering, NASA Langley Research Center, Hampton, VA, Technical Report No. NASA/CR-1998-208460, ICASE Report No. 98-33.
2.
Simpson
,
T. W.
,
Peplinski
,
J. D.
,
Koch
,
P. N.
, and
Allen
,
J. K.
, 2001, “
Metamodels for Computer-Based Engineering Design: Survey and Recommendations
,”
Eng. Comput.
,
17
, pp.
129
150
.
3.
Simpson
,
T. W.
,
Mauery
,
T. M.
,
Korte
,
J. J.
, and
Mistree
,
F.
, 2001, “
Kriging Models for Global Approximation in Simulation-Based Multidisciplinary Design Optimization
,”
AIAA J.
,
39
(
12
), pp.
2233
2241
.
4.
Jin
,
R.
,
Du
,
X.
, and
Chen
,
W.
, 2003, “
The Use of Metamodeling Techniques for Optimization Under Uncertainty
,”
Struct. Multidiscip. Optim.
,
25
, pp.
99
116
.
5.
Wang
,
G. G.
, and
Shan
,
S.
, 2007, “
Review of Metamodeling Techniques in Support of Engineering Design Optimization
,”
ASME J. Mech. Des.
,
129
(
4
), pp.
370
380
.
6.
Acar
,
E.
, and
Rais-Rohani
,
M.
, 2009, “
Ensemble of Metamodels With Optimized Weight Factors
,”
Struct. Multidiscip. Optim.
,
37
, pp.
279
294
.
7.
Forrester
,
A. I.
, and
Keane
,
A. J.
, 2009, “
Recent Advances in Surrogate-Based Optimization
,”
Prog. Aerosp. Sci.
,
45
(
1–3
), pp.
50
79
.
8.
Simpson
,
T.
,
Toropov
,
V.
,
Balabanov
,
V.
, and
Viana
,
F.
, 2008, “
Design and Analysis of Computer Experiments in Multidisciplinary Design Optimization: A Review of How far we Have Come–or Not
,”
12th AIAA/ISSMO Multidisciplinary Analysis and Optimization Conference
, Victoria, Canada, AIAA2008-5802, pp.
10
12
9.
Romero
,
D.
,
Amon
,
C. H.
, and
Finger
,
S.
, 2006, “
On Adaptive Sampling for Single and Multi-Response Bayesian Surrogate Models
,”
2006 ASME International Design Engineering Technical Conferences and Computers and Information in Engineering Conference (IDETC/CIE 2006)
, Philadelphia, Pennsylvania, Paper No. DETC2006-99210, pp.
10.
Myers
,
D. E.
, 1982, “
Matrix Formulation of Co-Kriging
,”
Math. Geol.
,
14
(
3
), pp.
249
257
.
11.
Myers
,
D. E.
, 1992, “
Kriging, Cokriging, Radial Basis Functions and the Role of Positive Definiteness
,”
Comput. Math. Appl.
,
24
(
12
), pp.
139
148
.
12.
Ver Hoef
,
J. M.
, and
Cressie
,
N.
, 1993, “
Multivariable Spatial Prediction
,”
Math. Geol.
,
25
(
2
), pp.
219
240
.
13.
Jin
,
R.
,
Chen
,
W.
, and
Simpson
,
T. W.
, 2001, “
Comparative Studies of Metamodeling Techniques Under Multiple Modeling Criteria
,”
Struct. Multidiscip. Optim.
,
23
, pp.
1
13
.
14.
Meckesheimer
,
M.
,
Barton
,
R. R.
,
Simpson
,
T.
,
Limayem
,
F.
, and
Yannou
,
B.
, 2001, “
Metamodeling of Combined Discrete/Continuous Responses
,”
AIAA J.
,
39
(
10
), pp.
1950
1959
.
15.
Sobieszczanski-Sobieski
,
J.
, and
Haftka
,
R. T.
, 1997, “
Multidisciplinary Aerospace Design Optimization: Survey of Recent Developments
,”
Struct. Optim.
,
14
, pp.
1
23
.
16.
Clarke
,
S.
,
Griebsch
,
J. H.
, and
Simpson
,
T.
, 2005, “
Analysis of Support Vector Regression for Approximation of Complex Engineering Analyses
,”
Trans. ASME J. Mech. Des.
,
127
(
6
), pp.
1077
1087
.
17.
Viana
,
F.
, and
Haftka
,
R.
, 2008, “
Using Multiple Surrogates for Metamodeling
,”
Proceedings of the 7th ASMO-UK/ISSMO International Conference on Engineering Design Optimization
,
Bath
,
United Kingdom
.
18.
Jones
,
D. R.
,
Schonlau
,
M.
, and
Welch
,
W. J.
, 1998, “
Efficient Global Optimization of Expensive Black-Box Functions
,”
J. Global Optim.
,
13
, pp.
455
492
.
19.
Ginsbourger
,
D.
,
Le Riche
,
R.
, and
Carraro
,
L.
, 2007, “
Kriging is Well-Suited to Parallelize Optimization
,”
Computational Intelligence in Expensive Optimization Problems: Adaptation, Learning and Optimization, Vol. 2
,
Y.
Tenne
and
C.-K.
Goh
, eds.,
Springer-Verlag Berlin Heidelberg
, pp.
131
162
.
20.
Ponweiser
,
W.
,
Wagner
,
T.
, and
Vincze
,
M.
, 2008, “
Clustered Multiple Generalized Expected Improvement: A Novel Infill Sampling Criterion for Surrogate Models
,”
IEEE Congress on Evolutionary Computation (CEC) 2008
, pp.
3515
3522
.
21.
Queipo
,
N.
,
Haftka
,
R.
,
Shyy
,
W.
,
Goel
,
T.
,
Vaidyanathan
,
R.
, and
Tucker
,
P. K.
, 2005, “
Surrogate-Based Analysis and Optimization
,”
Prog. Aerosp. Sci.
,
41
, pp.
1
28
.
22.
Li
,
G.
,
Azarm
,
S.
,
Farhang-Mehr
,
A.
, and
Diaz
,
A. R.
, 2006, “
Approximation of Multiresponse Deterministic Engineering Simulations: A Dependent Metamodeling Approach
,”
Struct. Multidiscip. Optim.
,
31
, pp.
260
269
.
23.
Fausett
,
L.
, 1998,
Fundamentals of Neural Networks: Architectures, Algorithms and Applications
,
Prentice Hall
,
Englewood Cliffs, NJ
.
24.
Hagan
,
M. T.
, and
Menhaj
,
M. B.
, 1994, “
Training Feedforward Networks With the Marquardt Algorithm
,”
IEEE Trans. Neural Networks
,
5
(
6
), pp.
989
992
.
25.
Rao
,
M. S.
,
Protopopescu
,
V.
,
Mann
,
R. C.
, and
Iyengar
,
S. S.
, 1996, “
Learning Algorithms for Feed Forward Networks Based on Finite Samples
,”
IEEE Trans. Neural Networks
,
7
(
4
), pp.
926
930
.
26.
Luo
,
L.
,
Ji
,
A.
,
Ma
,
G.
, and
Guo
,
C.
, 1998, “
Focusing on One Component Each Time—Comparison of Single and Multiple Component Prediction Algorithms in Artificial Neural Networks for X-Ray Fluorescence Analysis
,”
X-Ray Spectrom.
,
27
, pp.
17
22
.
27.
Schimidt
,
F.
,
Cornejo-Ponce
,
L.
,
Bueno
,
M. I.
, and
Poppi
,
R. J.
, 2003, “
Determination of Some Rare Earth Elements by Edxrf and Artificial Neural Networks
,”
X-Ray Spectrom.
,
32
, pp.
423
427
.
28.
Goovaerts
,
P.
, 1997,
Geostatistics for Natural Resources Evaluation, Applied Geostatistics
,
Oxford University
,
New York
.
29.
Olea
,
R. A.
, 1999,
Geostatistics for Engineers and Earth Scientists
,
Kluwer Academic Publishers
,
Norwell, MA
.
30.
Wackernagel
,
H.
, 1995,
Multivariate Geostatistics
,
Springer-Verlag
,
Berlin
.
31.
Cressie
,
N.
, 1991,
Statistics for Spatial Data, Probability and Mathematical Statistics
,
John Wiley & Sons
,
New York
, p.
900
.
32.
Osio
,
I. G.
, 1996, “
Multistage Bayesian Surrogates and Optimal Sampling for Engineering Design and Process Improvement
,” Ph.D. thesis, Department of Mechanical Engineering, Carnegie Mellon University, Pittsburgh, PA.
33.
Osio
,
I. G.
, and
Amon
,
C. H.
, 1996, “
An Engineering Design Methodology With Multistage Bayesian Surrogates and Optimal Sampling
,”
Res. Eng. Des.
,
8
, pp.
189
206
.
34.
Leoni
,
N.
, 1999, “
Integrating Information Sources into Global Models: A Surrogate Methodology for Product and Process Development
,” Ph.D. Thesis, Department of Mechanical Engineering, Carnegie Mellon University, Pittsburgh, PA.
35.
Leoni
,
N.
, and
Amon
,
C. H.
, 1997, “
Transient Thermal Design of Wearable Computers With Embedded Electronics Using Phase Change Materials
,”
ASME Heat Transfer Div.
,
343
(
5
), pp.
49
56
.
36.
Leoni
,
N.
, and
Amon
,
C. H.
, 2000, “
Bayesian Surrogates for Integrating Numerical, Analytical and Experimental Data: Application to Inverse Heat Transfer in Wearable Computers
,”
IEEE Trans. Compon. Packag. Technol.
,
23
(
1
), pp.
23
32
.
37.
Pacheco
,
J. E.
, 2003, “
A Methodology for Surrogate Model Building in the Engineering Design Process
,” Ph.D. thesis, Department of Mechanical Engineering, Carnegie Mellon University, Pittsburgh, PA, USA.
38.
Pacheco
,
J. E.
,
Amon
,
C. H.
, and
Finger
,
S.
, 2003, “
Bayesian Surrogates Applied to Conceptual Stages of the Engineering Design Process
,”
Trans. ASME J. Mech. Des.
,
125
, pp.
664
672
.
39.
Santner
,
T. J.
,
Williams
,
B. J.
, and
Notz
,
W. I.
, 2003,
The Design and Analysis of Computer Experiments, Springer Series in Statistics
,
Springer-Verlag, Inc.
,
New York
.
40.
Morris
,
M. D.
,
Mitchell
,
T. J.
, and
Ylvisaker
,
D.
, 1993, “
Bayesian Design and Analysis of Computer Experiments: Use of Derivatives in Surface Prediction
,”
Technometrics
,
35
(
3
), pp.
243
255
.
41.
Kennedy
,
M. C.
, and
O’hagan
,
A.
, 2000, “
Predicting the Output From a Complex Computer Code When Fast Approximations are Available
,”
Biometrika
,
87
(
1
), pp.
1
13
.
42.
Lehman
,
J. S.
, 2002, “
Sequential Design of Computer Experiments for Robust Parameter Design
,” Ph.D. thesis, Ohio State University, Columbus, OH.
43.
Davis
,
G. J.
, and
Morris
,
M. D.
, 1997, “
Six Factors Which Affect the Condition Number of Matrices Associated with Kriging
,”
Math. Geol.
,
29
(
5
), pp.
669
683
.
44.
Oliver
,
D. S.
, 1998, “
Calculation of the Inverse of the Covariance
,”
Math. Geol.
,
30
(
7
), pp.
911
933
.
45.
Sacks
,
J.
,
Welch
,
W. J.
,
Mitchell
,
T. J.
, and
Wynn
,
H. P.
, 1989, “
Design and Analysis of Computer Experiments
,”
Stat. Sci.
,
4
(
4
), pp.
409
435
.
46.
Cressie
,
N.
, and
Wikle
,
C. K.
, 1998, “
The Variance-Based Cross-Variogram: You Can Add Apples and Oranges
,”
Math. Geol.
,
30
(
7
), pp.
789
799
.
47.
Shewry
,
M. C.
, and
Wynn
,
H. P.
, 1987, “
Maximum Entropy Sampling
,”
J. Appl. Stat.
,
14
(
2
), pp.
165
170
.
48.
Jin
,
R.
,
Chen
,
W.
, and
Sudjianto
,
A.
, 2002, “
On Sequential Sampling for Global Metamodeling in Engineering Design
,”
2002 International Design Engineering Technical Conferences and Computers and Information in Engineering Conference (IDETC/CIE)
, Montreal, Canada, Paper No. DETC2002/DAC-34092.
49.
Kleijnen
,
J. P. C.
, and
Van Beers
,
W. C. M.
, 2004, “
Application-Driven Sequential Designs for Simulation Experiments: Kriging Metamodeling
,”
J. Oper. Res. Soc.
,
55
(
8
), pp.
876
883
.
50.
Turner
,
C. J.
,
Campbell
,
M. I.
, and
Crawford
,
R. H.
, 2004, “
Metamodel Defined Multidimensional Embedded Sequential Sampling Criteria
,”
International Design Engineering Technical Conferences and Computers and Information in Engineering Conference (IDETC/CIE, 2004)
, Salt Lake City, UT, USA, Paper No. DETC2004-57722.
51.
Romero
,
D.
,
Amon
,
C. H.
, and
Finger
,
S.
, 2010, “
Improving Multi-Response Metamodels With Upper/Lower Bound Information Using Multi-Stage, Non-Stationary Covariance Functions
,”
2010 International Design Engineering Technical Conferences and Computers and Information in Engineering Conference (IDETC/CIE)
, Montreal, Quebec, Canada, Paper No. DETC2010-29030, pp.
52.
Wasserman
,
L.
, 2006,
All of Statistics: A Concise Course in Statistical Inference, Springer Texts in Statistics
,
Springer-Verlag
,
New York
.
53.
Burnham
,
K.
, and
Anderson
,
D.
, 2004, “
Multimodel Selection: Understanding Aic and Bic in Model Selection
,” Sociological Methods & Research, Vol. 33(2) Colorado State University, Cooperative Fish and Wildlife Research Unit, Fort Collins, Colorado, pp.
261
304
.
54.
Burnham
,
K.
, and
Anderson
,
D.
, 2002,
Model Selection and Multimodel Inference: A Practical-Theoretical Approach
,
Springer-Verlag
,
New York
.
55.
Akaike
,
H.
, 1974, “
A New Look at the Statistical Model Identification
,”
IEEE Trans. Autom. Control
,
AC-19
(
6
), pp.
716
723
.
56.
Romero
,
D.
,
Finger
,
S.
and
Amon
,
C.
, 2003, “
Modeling Time-Dependent Systems Using Multi-Stage Bayesian Surrogates
,”
ASME 2003 International Mechanical Engineering Congress and Exposition (IMECE2003)
, Washington, DC, USA, Paper No. IMECE2003-55049.
57.
Romero
,
D.
,
Amon
,
C. H.
,
Finger
,
S.
, and
Verdinelli
,
I.
, 2004, “
Multi-Stage Bayesian Surrogates for the Design of Time-Dependent Systems
,”
2004 International Design Engineering Technical Conferences and Computers and Information in Engineering Conference (IDETC/CIE)
, Salt Lake City, UT, USA, Paper No. DETC2004-57510.
58.
Sasena
,
M.
,
Papalambros
,
P.
, and
Goovaerts
,
P.
, 2002, “
Exploration of Metamodeling Sampling Criteria for Constrained Global Optimization
,”
Eng. Optimiz.
,
34
, pp.
263
278
.
59.
Coleman
,
T.
,
Branch
,
M. A.
, and
Grace
,
A.
, 2002,
Matlab - Optimization Toolbox User’s Guide
,
The Mathworks, Inc.
,
Natick, MA, USA,
Version: 2.2 R13.
60.
Viana
,
F.
,
Haftka
,
R.
, and
Steffen
,
V.
, 2009, “
Multiple Surrogates: How Cross-Validation Errors Can Help us to Obtain the Best Predictor
,”
Struct. Multidiscip. Optim.
,
39
, pp.
439
457
.
61.
Botros
,
K. K.
,
Kibrya
,
G.
, and
Glover
,
A.
, 2002, “
A Demonstration of Artificial-Neural-Networks-Based Data Mining for Gas-Turbine-Driven Compressor Stations
,”
Trans. ASME, J. Eng. Gas Turbines Power
,
124
, pp.
284
297
.
62.
Smith
,
B. R.
,
Romero
,
D. A.
,
Agonafer
,
D.
,
Gu
,
J.
, and
Amon
,
C. H.
, 2005, “
Aerogel for Microsystems Thermal Insulation: System Design and Process Development
,”
2005 ASME Summer Heat Transfer Conference
,
San Francisco, CA, USA
.
63.
Sobol
,
I. M.
, 2001, “
Global Sensitivity Indices for Nonlinear Mathematical Models and Their Monte Carlo Estimates
,”
Math. Comput. Simul.
,
55
, pp.
271
280
.
64.
Oakley
,
J.
, and
O’hagan
,
A.
, 2004, “
Probabilistic Sensitivity Analysis of Complex Models: A Bayesian Approach
,”
J. R. Stat. Soc. Ser. B (Methodol.)
,
66
(
3
), pp.
751
769
.
65.
Kennedy
,
M. C.
, and
O’hagan
,
A.
, 2001, “
Bayesian Calibration of Computer Models
,”
J. R. Stat. Soc. Ser. B (Methodol.)
,
63
(
3
), pp.
425
464
.
66.
Kennedy
,
M. C.
, and
O’Hagan
,
A.
, 2000, “
Predicting the Output from a Complex Computer Code when Fast Approximations are Available
,”
Biometrika
,
87
(1)
, pp.
1
13
.
67.
Helterbrand
,
J. D.
, and
Cressie
,
N.
, 1994, “
Universal Cokriging under Intrinsic Corregionalization
,”
Math. Geol.
,
26
(
2
), pp.
105
226
.
68.
Moore
,
R.
,
Romero
,
D.
, and
Paredis
,
C.
, 2011, “
A Rational Design Approach to Gaussian Process Modeling for Variable Fidelity Models
,”
2011 International Design Engineering Technical Conferences and Computers and Information in Engineering Conference (IDETC/CIE)
, Washington, DC, USA, Paper No. DETC2011-48227.
69.
Zadeh
,
P.
,
Toropov
,
V.
, and
Wood
,
A.
, 2009, “
Metamodel-Based Collaborative Optimization Framework
,”
Struct. Multidiscip. Optim.
,
38
, pp.
103
115
.
You do not currently have access to this content.