The optimization of black-box models is a challenging task owing to the lack of analytic gradient information and structural information about the underlying function, and also due often to significant run times. A common approach to tackling such problems is the implementation of Bayesian global optimization techniques. However, these techniques often rely on surrogate modeling strategies that endow the approximation of the underlying expensive function with nonexistent features. Further, these techniques tend to push new queries away from previously queried design points, making it difficult to locate an optimum point that rests near a previous model evaluation. To overcome these issues, we propose a gold rush (GR) policy that relies on purely local information to identify the next best design alternative to query. The method employs a surrogate constructed pointwise, that adds no additional features to the approximation. The result is a policy that performs well in comparison to state of the art Bayesian global optimization methods on several benchmark problems. The policy is also demonstrated on a constrained optimization problem using a penalty method.

References

References
1.
Durantin
,
C.
,
Marzat
,
J.
, and
Balesdent
,
M.
,
2016
, “
Analysis of Multi-Objective Kriging-Based Methods for Constrained Global Optimization
,”
Comput. Optim. Appl.
,
63
(
3
), pp.
903
926
.
2.
Sasena
,
M. J.
,
Papalambros
,
P.
, and
Goovaerts
,
P.
,
2002
, “
Exploration of Metamodeling Sampling Criteria for Constrained Global Optimization
,”
Eng. Optim.
,
34
(
3
), pp.
263
278
.
3.
Ghoreishi
,
S. F.
, and
Allaire
,
D. L.
,
2018
, “
A Fusion-Based Multi-Information Source Optimization Approach Using Knowledge Gradient Policies
,”
AIAA
Paper No. 2018-1159.
4.
Parr
,
J.
,
Holden
,
C. M.
,
Forrester
,
A. I.
, and
Keane
,
A. J.
,
2010
, “
Review of Efficient Surrogate Infill Sampling Criteria With Constraint Handling
,”
Second International Conference on Engineering Optimization
, Lisbon, Portugal, Sept. 6–9, pp. 1–10.https://pdfs.semanticscholar.org/df63/8813760971a61daaa91259ff65b817d0dd51.pdf
5.
Shan
,
S.
, and
Wang
,
G. G.
,
2010
, “
Metamodeling for High Dimensional Simulation-Based Design Problems
,”
ASME J. Mech. Des.
,
132
(
5
), p.
051009
.
6.
Forrester
,
A. I.
, and
Keane
,
A. J.
,
2009
, “
Recent Advances in Surrogate-Based Optimization
,”
Prog. Aerosp. Sci.
,
45
(
1–3
), pp.
50
79
.
7.
Booker
,
A. J.
,
Dennis
,
J. E.
,
Frank
,
P. D.
,
Serafini
,
D. B.
,
Torczon
,
V.
, and
Trosset
,
M. W.
,
1999
, “
A Rigorous Framework for Optimization of Expensive Functions by Surrogates
,”
Struct. Optim.
,
17
(
1
), pp.
1
13
.
8.
MoŠkus J.,1975, “On Bayesian Methods for Seeking the Extremum,” Optimization Techniques IFIP Technical Conference, Novosibirsk, July 1–7, pp. 400–405.
9.
Žilinskas
,
A.
,
1992
, “
A Review of Statistical Models for Global Optimization
,”
J. Global Optim.
,
2
(
2
), pp.
145
153
.
10.
Rasmussen
,
C. E.
,
2004
, “
Gaussian Processes in Machine Learning
,”
Advanced Lectures on Machine Learning. ML 2003. Lecture Notes in Computer Science
, In: Bousquet O., von Luxburg U., Rätsch G., eds., Vol. 3176, Springer, Berlin, pp.
63
71
.
11.
Osborne
,
M. A.
,
Garnett
,
R.
, and
Roberts
,
S. J.
,
2009
, “
Gaussian Processes for Global Optimization
,”
Third International Conference on Learning and Intelligent Optimization (LION3), Trento, Italy, Jan. 14–18
, pp.
1
15
.
12.
Pandita
,
P.
,
Bilionis
,
I.
, and
Panchal
,
J.
,
2016
, “
Extending Expected Improvement for High-Dimensional Stochastic Optimization of Expensive Black-Box Functions
,”
ASME J. Mech. Des.
,
138
(
11
), p.
111412
.
13.
Koullias
,
S.
, and
Mavris
,
D. N.
,
2014
, “
Methodology for Global Optimization of Computationally Expensive Design Problems
,”
ASME J. Mech. Des.
,
136
(
8
), p.
081007
.
14.
Jones
,
D. R.
,
Schonlau
,
M.
, and
Welch
,
W. J.
,
1998
, “
Efficient Global Optimization of Expensive Black-Box Functions
,”
J. Global Optim.
,
13
(
4
), pp.
455
492
.
15.
Huang
,
D.
,
Allen
,
T. T.
,
Notz
,
W. I.
, and
Miller
,
R. A.
,
2006
, “
Sequential Kriging Optimization Using Multiple-Fidelity Evaluations
,”
Struct. Multidiscip. Optim.
,
32
(
5
), pp.
369
382
.
16.
Humphrey
,
D. G.
, and
Wilson
,
J. R.
,
2000
, “
A Revised Simplex Search Procedure for Stochastic Simulation Response Surface Optimization
,”
INFORMS J. Comput.
,
12
(
4
), pp.
272
283
.
17.
Borror
,
C. M.
,
Montgomery
,
D. C.
, and
Myers
,
R. H.
,
2002
, “
Evaluation of Statistical Designs for Experiments Involving Noise Variables
,”
J. Qual. Technol.
,
34
(
1
), p.
54
.
18.
Gablonsky
,
J. M.
, and
Kelley
,
C. T.
,
2001
, “
A Locally-Biased Form of the Direct Algorithm
,”
J. Global Optim.
,
21
(
1
), pp.
27
37
.
19.
Villemonteix
,
J.
,
Vazquez
,
E.
, and
Walter
,
E.
,
2009
, “
An Informational Approach to the Global Optimization of Expensive-to-Evaluate Functions
,”
J. Global Optim.
,
44
(
4
), p.
509
.
20.
Kushner
,
H. J.
, and
Schweppe
,
F. C.
,
1964
, “
A Maximum Principle for Stochastic Control Systems
,”
J. Math. Anal. Appl.
,
8
(
2
), pp.
287
302
.
21.
Jones
,
D. R.
,
2001
, “
A Taxonomy of Global Optimization Methods Based on Response Surfaces
,”
J. Global Optim.
,
21
(
4
), pp.
345
383
.
22.
Shimoyama
,
K.
,
Sato
,
K.
,
Jeong
,
S.
, and
Obayashi
,
S.
,
2013
, “
Updating Kriging Surrogate Models Based on the Hypervolume Indicator in Multi-Objective Optimization
,”
ASME J. Mech. Des.
,
135
(
9
), p.
094503
.
23.
Brochu
,
E.
,
Cora
,
V. M.
, and
De Freitas
,
N.
,
2010
, “
A Tutorial on Bayesian Optimization of Expensive Cost Functions, With Application to Active User Modeling and Hierarchical Reinforcement Learning
,” e-print
arXiv:1012.2599
.https://arxiv.org/abs/1012.2599
24.
Lai
,
T. L.
, and
Robbins
,
H.
,
1985
, “
Asymptotically Efficient Adaptive Allocation Rules
,”
Adv. Appl. Math.
,
6
(
1
), pp.
4
22
.
25.
Hernández-Lobato
,
J. M.
,
Hoffman
,
M. W.
, and
Ghahramani
,
Z.
,
2014
, “
Predictive Entropy Search for Efficient Global Optimization of Black-Box Functions
,”
Adv. Neutral Inf. Process. Syst.
, 1, pp. 918–926. https://pdfs.semanticscholar.org/fbce/d739237b1bd07a6f3fb627b4c948751ca659.pdf
26.
Moore
,
R. A.
,
Romero
,
D. A.
, and
Paredis
,
C. J.
,
2014
, “
Value-Based Global Optimization
,”
ASME J. Mech. Des.
,
136
(
4
), p.
041003
.
27.
Thompson
,
S. C.
, and
Paredis
,
C. J.
,
2010
, “
An Investigation Into the Decision Analysis of Design Process Decisions
,”
ASME J. Mech. Des.
,
132
(
12
), p.
121009
.
28.
Frazier
,
P. I.
,
Powell
,
W. B.
, and
Dayanik
,
S.
,
2008
, “
A Knowledge-Gradient Policy for Sequential Information Collection
,”
SIAM J. Control Optim.
,
47
(
5
), pp.
2410
2439
.
29.
Gupta
,
S. S.
, and
Miescke
,
K. J.
,
1996
, “
Bayesian Look Ahead One-Stage Sampling Allocations for Selection of the Best Population
,”
J. Stat. Plann. Inference
,
54
(
2
), pp.
229
244
.
30.
Frazier
,
P.
,
Powell
,
W.
, and
Dayanik
,
S.
,
2009
, “
The Knowledge-Gradient Policy for Correlated Normal Beliefs
,”
INFORMS J. Comput.
,
21
(
4
), pp.
599
613
.
31.
Negoescu
,
D. M.
,
Frazier
,
P. I.
, and
Powell
,
W. B.
,
2011
, “
The Knowledge-Gradient Algorithm for Sequencing Experiments in Drug Discovery
,”
INFORMS J. Comput.
,
23
(
3
), pp.
346
363
.
32.
Wu
,
J.
,
Poloczek
,
M.
,
Wilson
,
A. G.
, and
Frazier
,
P.
,
2017
, “
Bayesian Optimization With Gradients
,” eprint
arXiv:1703.04389
https://arxiv.org/abs/1703.04389
33.
Andrianakis
,
I.
, and
Challenor
,
P. G.
,
2012
, “
The Effect of the Nugget on Gaussian Process Emulators of Computer Models
,”
Comput. Stat. Data Anal.
,
56
(
12
), pp.
4215
4228
.
34.
Vapnik
,
V.
,
1998
,
Statistical Learning Theory
,
Wiley
,
New York
.
35.
Jakeman
,
J. D.
, and
Wildey
,
T.
,
2015
, “
Enhancing Adaptive Sparse Grid Approximations and Improving Refinement Strategies Using Adjoint-Based a Posteriori Error Estimates
,”
J. Comput. Phys.
,
280
, pp.
54
71
.
36.
Rabitz
,
H.
, and
Aliş
,
Ö. F.
,
1999
, “
General Foundations of High-Dimensional Model Representations
,”
J. Math. Chem.
,
25
(
2/3
), pp.
197
233
.
37.
Li
,
K.
, and
Allaire
,
D.
,
2016
, “
A Compressed Sensing Approach to Uncertainty Propagation for Approximately Additive Functions
,”
ASME
Paper No. DETC2016-60195
.
38.
Gorodetsky
,
A. A.
,
Karaman
,
S.
, and
Marzouk
,
Y. M.
,
2015
, “
Function-Train: A Continuous Analogue of the Tensor-Train Decomposition
,” eprint
arXiv:1510.09088
.https://arxiv.org/abs/1510.09088
39.
Allaire
,
D.
, and
Willcox
,
K.
,
2010
, “
Surrogate Modeling for Uncertainty Assessment With Application to Aviation Environmental System Models
,”
AIAA J.
,
48
(
8
), pp.
1791
1803
.
40.
Amaral
,
S.
,
Allaire
,
D.
, and
Willcox
,
K.
,
2017
, “
Optimal L2-norm Empirical Importance Weights for the Change of Probability Measure
,”
Stat. Comput.
,
27
(
3
), pp.
625
643
.
41.
Altman
,
N. S.
,
1992
, “
An Introduction to Kernel and Nearest-Neighbor Nonparametric Regression
,”
Am. Statistician
,
46
(
3
), pp.
175
185
.
42.
Molga
,
M.
, and
Smutnicki
,
C.
,
2005
, “
Test Functions for Optimization Needs
,”
Comput. Inform. Sci.
,
01
, pp.
1
43
.http://new.zsd.iiar.pwr.wroc.pl/files/docs/functions.pdf
43.
Branin
,
F. H.
,
1972
, “
Widely Convergent Method for Finding Multiple Solutions of Simultaneous Nonlinear Equations
,”
IBM J. Res. Dev.
,
16
(
5
), pp.
504
522
.
44.
Huang
,
D.
,
Allen
,
T. T.
,
Notz
,
W. I.
, and
Zeng
,
N.
,
2006
, “
Global Optimization of Stochastic Black-Box Systems Via Sequential Kriging Meta-Models
,”
J. Global Optim.
,
34
(
3
), pp.
441
466
.
45.
Ray
,
T.
,
2003
, “
Golinski's Speed Reducer Problem Revisited
,”
AIAA J.
,
41
(
3
), pp.
556
558
.
46.
Hassan
,
R.
,
Cohanim
,
B.
,
De Weck
,
O.
, and
Venter
,
G.
,
2005
, “
A Comparison of Particle Swarm Optimization and the Genetic Algorithm
,”
AIAA
Paper No. 2005-1897.
47.
Li
,
H.
, and
Papalambros
,
P.
,
1985
, “
A Production System for Use of Global Optimization Knowledge
,”
J. Mech., Transm., Autom. Des.
,
107
(
2
), pp.
277
284
.
48.
Ku
,
K. J.
,
Rao
,
S.
, and
Chen
,
L.
,
1998
, “
Taguchi-Aided Search Method for Design Optimization of Engineering Systems
,”
Eng. Optim.
,
30
(
1
), pp.
1
23
.
49.
Kanukolanu
,
D.
,
Lewis
,
K. E.
, and
Winer
,
E. H.
,
2006
, “
A Multidimensional Visualization Interface to Aid in Trade-Off Decisions During the Solution of Coupled Subsystems Under Uncertainty
,”
ASME J. Comput. Inf. Sci. Eng.
,
6
(
3
), pp.
288
299
.
50.
McKay
,
M. D.
,
Beckman
,
R. J.
, and
Conover
,
W. J.
,
1979
, “
Comparison of Three Methods for Selecting Values of Input Variables in the Analysis of Output From a Computer Code
,”
Technometrics
,
21
(
2
), pp.
239
245
.
51.
De Boer
,
P.-T.
,
Kroese
,
D. P.
,
Mannor
,
S.
, and
Rubinstein
,
R. Y.
,
2005
, “
A Tutorial on the Cross-Entropy Method
,”
Ann. Oper. Res.
,
134
(
1
), pp.
19
67
.
52.
Glover
,
F.
,
1989
, “
Tabu Search—Part I
,”
ORSA J. Comput.
,
1
(
3
), pp.
190
206
.
You do not currently have access to this content.