Abstract

The cost of adaptive sampling for global metamodeling depends on the total number of costly function evaluations and to which degree these evaluations are performed in parallel. Conventionally, samples are taken through a greedy sampling strategy that is optimal for either a single sample or a handful of samples. The limitation of such an approach is that they compromise optimality when more samples are taken. In this paper, we propose a thrifty adaptive batch sampling (TABS) approach that maximizes a multistage reward function to find an optimal sampling policy containing the total number of sampling stages, the number of samples per stage, and the spatial location of each sample. Consequently, the first batch identified by TABS is optimal with respect to all potential future samples, the available resources, and is consistent with a modeler’s preference and risk attitude. Moreover, we propose two heuristic-based strategies that reduce numerical complexity with a minimal reduction in optimality. Through numerical examples, we show that TABS outperforms or is comparable with greedy sampling strategies. In short, TABS provides modelers with a flexible adaptive sampling tool for global metamodeling that effectively reduces sampling costs while maintaining prediction accuracy.

References

References
1.
Liu
,
R.
,
Yabansu
,
Y. C.
,
Yang
,
Z.
,
Choudhary
,
A. N.
,
Kalidindi
,
S. R.
, and
Agrawal
,
A.
,
2017
, “
Context Aware Machine Learning Approaches for Modeling Elastic Localization in Three-Dimensional Composite Microstructures
,”
Integr. Mater. Manuf. Innov.
,
6
(
2
), pp.
160
171
. 10.1007/s40192-017-0094-3
2.
Zhang
,
S.
,
Zhu
,
P.
, and
Chen
,
W.
,
2013
, “
Crashworthiness-Based Lightweight Design Problem via New Robust Design Method Considering Two Sources of Uncertainties
,”
J. Mech. Eng. Sci.
,
227
(
7
), pp.
1381
1391
. 10.1177/0954406212460824
3.
Bessa
,
M. A.
,
Bostanabad
,
R.
,
Liu
,
Z.
,
Hu
,
A.
,
Apley
,
D. W.
,
Brinson
,
C.
,
Chen
,
W.
, and
Liu
,
W. K.
,
2017
, “
A Framework for Data-Driven Analysis of Materials Under Uncertainty: Countering the Curse of Dimensionality
,”
Comput. Methods Appl. Mech. Eng.
,
320
(
1
), pp.
633
667
. 10.1016/j.cma.2017.03.037
4.
Viana
,
F. A. C.
,
Gogu
,
C.
, and
Haftka
,
R. T.
,
2010
, “
Making the Most Out of Surrogate Models: Tricks of the Trade
,”
Proceedings of the ASME 2010 International Design Engineering Technical Conferences and Computers and Information in Engineering Conference
,
Montreal, Canada
,
Aug. 15–18
, pp.
587
598
.
5.
Santner
,
T. J.
,
Williams
,
B. J.
, and
Notz
,
W. I.
,
2003
,
The Design and Analysis of Computer Experiments
,
Springer-Verlag
,
New York
.
6.
Garbo
,
A.
, and
German
,
B.
,
2017
, “
Adaptive Sampling With Adaptive Surrogate Model Selection for Computer Experiment Applications
,”
AIAA/ISSMO Multidisciplinary Analysis and Optimization Conference
,
Denver, CO
,
July 5–9
, pp.
1
26
.
7.
Jin
,
R.
,
Chen
,
W.
, and
Simpson
,
T. W.
,
2001
, “
Comparative Studies of Metamodelling Techniques Under Multiple Modelling Criteria
,”
Struct. Multidiscip. Optim.
,
23
(
1
), pp.
1
13
. 10.1007/s00158-001-0160-4
8.
Jin
,
R.
,
Chen
,
W.
, and
Sudjianto
,
A.
,
2005
, “
An Efficient Algorithm for Constructing Optimal Design of Computer Experiments
,”
J. Stat. Plan. Inference
,
134
(
1
), pp.
268
287
. 10.1016/j.jspi.2004.02.014
9.
Olsson
,
A.
,
Sandberg
,
G.
, and
Dahlblom
,
O.
,
2003
, “
On Latin Hypercube Sampling for Structural Reliability Analysis
,”
Struct. Saf.
,
25
(
1
), pp.
47
68
. 10.1016/S0167-4730(02)00039-5
10.
Jin
,
R.
,
Chen
,
W.
, and
Sudjianto
,
A.
,
2002
, “
On Sequential Sampling for Global Metamodeling in Engineering Design
,”
Design Engineering Technical Conferences and Computers and Information in Engineering
,
Montreal, Canada
,
Sept. 29–Oct. 2
, pp.
1
10
.
11.
Lam
,
R.
,
Allaire
,
D. L.
, and
Willcox
,
K. E.
,
2015
, “
Multifidelity Optimization Using Statistical Surrogate Modeling for Non-Hierarchical Information Sources
,”
Am. Inst. Aeronaut. Astronaut.
,
56
(
1
), pp.
1
21
.
12.
Forrester
,
A. I. J.
, and
Keane
,
A. J.
,
2009
, “
Recent Advances in Surrogate-Based Optimization
,”
Prog. Aerosp. Sci.
,
45
(
1–3
), pp.
50
79
. 10.1016/j.paerosci.2008.11.001
13.
Kandasamy
,
K.
,
Schneider
,
J.
, and
Poczos
,
B.
,
2015
, “
High Dimensional Bayesian Optimisation and Bandits via Additive Models
,”
International Conference on Machine Learning
,
Lille, France
,
July 6–11
, pp.
1
10
.
14.
Jones
,
D.
,
2001
, “
A Taxonomy of Global Optimization Methods Based on Response Surfaces
,”
J. Glob. Optim.
,
21
(
4
), pp.
345
383
. 10.1023/A:1012771025575
15.
Viana
,
F. A. C.
,
Haftka
,
R. T.
, and
Watson
,
L. T.
,
2013
, “
Efficient Global Optimization Algorithm Assisted by Multiple Surrogate Techniques
,”
J. Glob. Optim.
,
56
(
2
), pp.
669
689
. 10.1007/s10898-012-9892-5
16.
Zhao
,
L.
,
Choi
,
K.
, and
Lee
,
I.
,
2010
, “
A Metamodeling Method Using Dynamic Kriging and Sequential Sampling
,”
13th AIAA/ISSMO Multidisciplinary Analysis and Optimization Conference
,
Fort Worth, TX
,
Sept. 13–15
, pp.
1
18
.
17.
Marques
,
A. N.
,
Lam
,
R. R.
, and
Willcox
,
K. E.
,
2018
, “
Contour Location via Entropy Reduction Leveraging Multiple Information Sources
,”
Conference on Neural Information Processing Systems
,
Montreal, Canada
,
Dec. 3–8
, pp.
1
11
.
18.
Ranjan
,
P.
,
Bingham
,
D.
, and
Michailidis
,
G.
,
2008
, “
Sequential Experiment Design for Contour Estimation From Complex Computer Codes
,”
Technometrics
,
50
(
4
), pp.
527
541
. 10.1198/004017008000000541
19.
Picheny
,
V.
,
Ginsbourger
,
D.
,
Roustant
,
O.
,
Haftka
, and
RaphaelKim
,
N.
,
2010
, “
Adaptive Designs of Experiments for Accurate Approximation of a Target Region
,”
ASME J. Mech. Des.
,
132
(
7
), pp.
1
13
. 10.1115/1.4001873
20.
Jiang
,
Z.
,
Chen
,
S.
,
Apley
,
D. W.
, and
Chen
,
W.
,
2016
, “
Reduction of Epistemic Model Uncertainty in Simulation-Based Multidisciplinary Design
,”
ASME J. Mech. Des.
,
138
(
8
), p.
8
. 10.1115/1.4033918
21.
Xu
,
S.
,
Liu
,
H.
,
Wang
,
X.
, and
Jiang
,
X.
,
2014
, “
A Robust Error-Pursuing Sequential Sampling Approach for Global Metamodeling Based on Voronoi Diagram and Cross Validation
,”
ASME J. Mech. Des.
,
136
(
7
), pp.
1
10
. 10.1115/1.4027161
22.
Liu
,
Y.
,
Shi
,
Y.
,
Zhou
,
Q.
, and
Xiu
,
R.
,
2016
, “
A Sequential Sampling Strategy to Improve the Global Fidelity of Metamodels in Multi-Level System Design
,”
Struct. Multidiscip. Optim.
,
53
(
6
), pp.
1295
1313
. 10.1007/s00158-015-1379-9
23.
Sacks
,
J.
,
Welch
,
W. J.
,
Mitchell
,
T. J.
, and
Wynn
,
H. P.
,
1989
, “
Design and Analysis of Computer Experiments
,”
Stat. Sci.
,
4
(
4
), pp.
409
423
. 10.1214/ss/1177012413
24.
Kushner
,
H. J.
,
1964
, “
A New Method of Locating the Maximum Point of an Arbitrary Multipeak Curve in the Presence of Noise
,”
J. Basic Eng.
,
86
(
1
), pp.
97
106
. 10.1115/1.3653121
25.
Chen
,
S.
,
Jiang
,
Z.
,
Yang
,
S.
, and
Chen
,
W.
,
2016
, “
Multimodel Fusion Based Sequential Optimization
,”
AIAA J.
,
55
(
1
), pp.
241
254
. 10.2514/1.J054729
26.
Huang
,
D.
,
Allen
,
T. T.
,
Notz
,
W. I.
, and
Miller
,
R. A.
,
2006
, “
Sequential Kriging Optimization Using Multiple Fidelity Evaluations
,”
Struct. Multidiscip. Optim.
,
32
(
5
), pp.
369
382
. 10.1007/s00158-005-0587-0
27.
Welch
,
W. J.
,
1983
, “
A Mean Squared Error Criterion for the Design of Experiments
,”
Biometrika
,
70
(
1
), pp.
205
213
. 10.1093/biomet/70.1.205
28.
Pronzato
,
L.
, and
Müller
,
W. G.
,
2012
, “
Design of Computer Experiments: Space Filling and Beyond
,”
Stat. Comput.
,
22
(
3
), pp.
681
701
. 10.1007/s11222-011-9242-3
29.
Binois
,
M.
,
Huang
,
J.
,
Gramacy
,
R. B.
, and
Ludkovski
,
M.
,
2018
, “
Replication or Exploration ? Sequential Design for Stochastic Simulation Experiments
,”
Technometrics
,
61
(
1
), pp.
1
17
.
30.
Loeppky
,
J. L.
,
Moore
,
L. M.
, and
Williams
,
B. J.
,
2010
, “
Batch Sequential Designs for Computer Experiments
,”
J. Stat. Plan. Inference
,
140
(
6
), pp.
1452
1464
. 10.1016/j.jspi.2009.12.004
31.
Lam
,
R.
,
Willcox
,
K.
, and
Wolpert
,
D. H.
,
2016
, “
Bayesian Optimization With a Finite Budget: An Approximate Dynamic Programming Approach
,”
Neural Inf. Process. Syst.
,
30
, pp.
883
891
.
32.
Wiering
,
M.
, and
van Otterlo
,
M.
,
2012
,
Reinforcement Learning
,
Springer-Verlag
,
Berlin
.
33.
Lam
,
R. R.
, and
Willcox
,
K. E.
,
2017
, “
Lookahead Bayesian Optimization With Inequality Constraints
,”
Advances in Neural Information Processing Systems
,
Long Beach, United States
,
Dec. 4–9
, pp.
1
11
.
34.
Kocis
,
L.
, and
Whiten
,
W. J.
,
1997
, “
Computational Investigations of Low-Discrepancy Sequences
,”
ACM Trans. Math. Softw.
,
23
(
2
), pp.
266
294
. 10.1145/264029.264064
35.
Hazelrigg
,
G. A.
,
1998
, “
A Framework for Decision-Based Engineering Design
,”
ASME J. Mech. Des.
,
120
(
4
), pp.
653
658
. 10.1115/1.2829328
36.
Apley
,
D. W.
,
Liu
,
J.
, and
Chen
,
W.
,
2006
, “
Understanding the Effects of Model Uncertainty in Robust Design With Computer Experiments
,”
ASME J. Mech. Des.
,
128
(
4
), pp.
945
958
. 10.1115/1.2204974
37.
Bertsimas
,
D.
, and
Thiele
,
A.
,
2006
, “
Robust and Data-Driven Optimization: Modern Decision Making Under Uncertainty
,”
Model. Methods, Appl. Innov. Decis. Mak.
,
36
, pp.
95
122
.
38.
von Neumann
,
J.
, and
Morgenstern
,
O.
,
1953
,
Theory of Games and Economic Behavior
,
Princeton University Press
,
Princeton
.
39.
Abbas
,
A. E.
,
2006
, “
Maximum Entropy Utility
,”
Oper. Res.
,
54
(
2
), pp.
277
290
. 10.1287/opre.1040.0204
40.
Hazelrigg
,
G. A.
,
2012
,
Fundamentals of Decision Making
,
Neils Corp
,
Hammond, LA
.
41.
Gallager
,
R. G.
,
2013
,
Stochastic Processes: Theory for Applications
,
Cambridge University Press
,
New York
.
42.
Rasmussen
,
C. E.
, and
Williams
,
C. K. I.
,
2006
,
Gaussian Processes for Machine Learning
,
The MIT Press
,
Cambridge, Massachusetts
.
43.
Martin
,
J. D.
, and
Simpson
,
T. W.
,
2004
, “
On the Use of Kriging Models to Approximate Deterministic Computer Models
,”
Design Engineering Technical Conferences and Computers and Information in Engineering
,
Salt Lake City, UT
,
Sept. 28–Oct. 2
, pp.
1
12
.
44.
Tao
,
S.
,
Shintani
,
K.
,
Bostanabad
,
R.
,
Chan
,
Y.
,
Yang
,
G.
,
Meingast
,
H.
, and
Chen
,
W.
,
2017
, “
Enhanced Gaussian Process Metamodeling and Collaborative Optimization for Vehicle Suspension Design Optimization
,”
Design Engineering Technical Conferences And Computers and Information in Engineering
,
Cleveland, OH
,
Aug. 6–9
, pp.
1
12
.
45.
Bertsekas
,
D. P.
,
1995
,
Dynamic Programming and Optimal Control
,
Athena Scientific
,
Belmont
.
46.
Johnson
,
M. E.
,
Moore
,
L. M.
, and
Ylvisaker
,
D.
,
1990
, “
Minimax and Maximin Distance Designs
,”
J. Stat.
,
26
(
2
), pp.
131
148
.
47.
Bradley
,
S. P.
,
Hax
,
A. C.
, and
Magnanti
,
T. L.
,
1977
,
Applied Mathematical Programming
,
Addison-Wesley Publishing Company
,
Boston
.
48.
Shang
,
B.
, and
Apley
,
W. D.
,
2019
, “
Fully-Sequential Space-Filling Design Algorithms for Computer Experiments
,”
J. Qual. Technol.
https://doi.org/10.1080/00224065.2019.1705207
49.
Jiang
,
Z.
,
Apley
,
D. W.
, and
Chen
,
W.
,
2015
, “
Surrogate Preposterior Analyses for Predicting and Enhancing Identifiability in Model Calibration
,”
Int. J. Uncertain. Quantif.
,
5
(
4
), pp.
341
359
. 10.1615/Int.J.UncertaintyQuantification.2015012627
50.
Hogg
,
R. V.
,
McKean
,
J.
, and
Craig
,
A. T.
,
2014
,
Introduction to Mathematical Statistics
,
Pearson
,
London
.
51.
Mardia
,
K. V.
, and
Marshall
,
R.
,
1984
, “
Maximum Likelihood Estimation of Models for Residual Covariance in Spatial Regression
,”
Biometrika
,
71
(
1
), pp.
135
146
. 10.1093/biomet/71.1.135
52.
Abt
,
M.
, and
Welch
,
W. J.
,
1998
, “
Fisher Information and Maximum- Likelihood Estimation of Covariance Parameters in Gaussian Stochastic Processes
,”
Can. J. S
,
26
(
1
), pp.
127
137
. 10.2307/3315678
53.
van der Vaart
,
A.
,
1998
,
Asymptotic Statistics
,
Cambridge University Press
,
Cambridge
.
54.
Forrester
,
I. J.
,
Sobester
,
A.
, and
Keane
,
A. J.
,
2008
,
Engineering Design via Surrogate Modelling : A Practical Guide
,
Wiley
,
New York
.
55.
Jin
,
R.
,
Chen
,
W.
, and
Sudjianto
,
A.
,
2004
, “
Analytical Metamodel-Based Global Sensitivity Analysis and Uncertainty Propagation for Robust Design
,”
Soc. Automot. Eng.
,
1
(
429
), pp.
1
10
.
You do not currently have access to this content.