Abstract

Bayesian optimization is a metamodel-based global optimization approach that can balance between exploration and exploitation. It has been widely used to solve single-objective optimization problems. In engineering design, making trade-offs between multiple conflicting objectives is common. In this work, a multi-objective Bayesian optimization approach is proposed to obtain the Pareto solutions. A novel acquisition function is proposed to determine the next sample point, which helps improve the diversity and convergence of the Pareto solutions. The proposed approach is compared with some state-of-the-art metamodel-based multi-objective optimization approaches with four numerical examples and one engineering case. The results show that the proposed approach can obtain satisfactory Pareto solutions with significantly reduced computational cost.

References

References
1.
Aute
,
V.
,
Saleh
,
K.
,
Abdelaziz
,
O.
,
Azarm
,
S.
, and
Radermacher
,
R.
,
2013
, “
Cross-Validation Based Single Response Adaptive Design of Experiments for Kriging Metamodeling of Deterministic Computer Simulations
,”
Struct. Multidiscipl. Optim.
,
48
(
3
), pp.
581
605
. 10.1007/s00158-013-0918-5
2.
Kleijnen
,
J. P.
,
2009
, “
Kriging Metamodeling in Simulation: A Review
,”
Eur. J. Oper. Res.
,
192
(
3
), pp.
707
716
. 10.1016/j.ejor.2007.10.013
3.
Xiao
,
M.
,
Gao
,
L.
,
Shao
,
X.
,
Qiu
,
H.
, and
Jiang
,
P.
,
2012
, “
A Generalised Collaborative Optimisation Method and Its Combination With Kriging Metamodels for Engineering Design
,”
J. Eng. Des.
,
23
(
5
), pp.
379
399
. 10.1080/09544828.2011.595706
4.
Fang
,
H.
, and
Horstemeyer
,
M. F.
,
2006
, “
Global Response Approximation With Radial Basis Functions
,”
Eng. Optim.
,
38
(
4
), pp.
407
424
. 10.1080/03052150500422294
5.
Can
,
B.
, and
Heavey
,
C.
,
2012
, “
A Comparison of Genetic Programming and Artificial Neural Networks in Metamodeling of Discrete-Event Simulation Models
,”
Comput. Oper. Res.
,
39
(
2
), pp.
424
436
. 10.1016/j.cor.2011.05.004
6.
Kleijnen
,
J. P.
,
2008
, “
Response Surface Methodology for Constrained Simulation Optimization: An Overview
,”
Simul. Modell. Pract. Theory
,
16
(
1
), pp.
50
64
. 10.1016/j.simpat.2007.10.001
7.
Li
,
M.
,
Li
,
G.
, and
Azarm
,
S.
,
2008
, “
A Kriging Metamodel Assisted Multi-Objective Genetic Algorithm for Design Optimization
,”
ASME J. Mech. Des.
,
130
(
3
), p.
031401
. 10.1115/1.2829879
8.
Zhao
,
L.
,
Choi
,
K. K.
,
Lee
,
I.
, and
Gorsich
,
D.
,
2013
, “
Conservative Surrogate Model Using Weighted Kriging Variance for Sampling-Based RBDO
,”
ASME J. Mech. Des.
,
135
(
9
), p.
091003
. 10.1115/1.4024731
9.
Hu
,
Z.
, and
Du
,
X.
,
2015
, “
Mixed Efficient Global Optimization for Time-Dependent Reliability Analysis
,”
ASME J. Mech. Des.
,
137
(
5
), p.
051401
. 10.1115/1.4029520
10.
Li
,
M.
, and
Wang
,
Z.
,
2018
, “
Confidence-Driven Design Optimization Using Gaussian Process Metamodeling With Insufficient Data
,”
ASME J. Mech. Des.
,
140
(
12
), p.
121405
. 10.1115/1.4040985
11.
Moustapha
,
M.
,
Sudret
,
B.
,
Bourinet
,
J.-M.
, and
Guillaume
,
B.
,
2016
, “
Quantile-Based Optimization Under Uncertainties Using Adaptive Kriging Surrogate Models
,”
Struct. Multidiscipl. Optim.
,
54
(
6
), pp.
1403
1421
. 10.1007/s00158-016-1504-4
12.
Zhang
,
S.
,
Zhu
,
P.
,
Chen
,
W.
, and
Arendt
,
P.
,
2013
, “
Concurrent Treatment of Parametric Uncertainty and Metamodeling Uncertainty in Robust Design
,”
Struct. Multidiscipl. Optim.
,
47
(
1
), pp.
63
76
. 10.1007/s00158-012-0805-5
13.
Schonlau
,
M.
,
1997
, “
Computer Experiments and Global Optimization
,”
Ph.D. thesis
,
University of Waterloo
,
Waterloo
.
14.
Jones
,
D. R.
,
2001
, “
A Taxonomy of Global Optimization Methods Based on Response Surfaces
,”
J. Global Optim.
,
21
(
4
), pp.
345
383
. 10.1023/A:1012771025575
15.
Couckuyt
,
I.
,
Deschrijver
,
D.
, and
Dhaene
,
T.
,
2014
, “
Fast Calculation of Multiobjective Probability of Improvement and Expected Improvement Criteria for Pareto Optimization
,”
J. Global Optim.
,
60
(
3
), pp.
575
594
. 10.1007/s10898-013-0118-2
16.
Snoek
,
J.
,
Larochelle
,
H.
, and
Adams
,
R. P.
,
2012
, “
Practical Bayesian Optimization of Machine Learning Algorithms
,”
Proceedings of Advances in Neural Information Processing Systems
,
Lake Tahoe, NV
,
Dec. 3–6
, pp.
2951
2959
.
17.
Zheng
,
J.
,
Li
,
Z.
,
Gao
,
L.
, and
Jiang
,
G.
,
2016
, “
A Parameterized Lower Confidence Bounding Scheme for Adaptive Metamodel-Based Design Optimization
,”
Eng. Comput.
,
33
(
7
), pp.
2165
2184
. 10.1108/EC-04-2015-0088
18.
Liu
,
B.
,
Zhang
,
Q.
,
Fernández
,
F. V.
, and
Gielen
,
G.
,
2012
, “
Self-Adaptive Lower Confidence Bound: A New General and Effective Prescreening Method for Gaussian Process Surrogate Model Assisted Evolutionary Algorithms
,”
Proceedings of 2012 IEEE Congress on Evolutionary Computation (CEC)
,
Brisbane, Australia
,
June 10–15
,
IEEE
, pp.
1
6
.
19.
Martinez-Cantin
,
R.
,
de Freitas
,
N.
,
Doucet
,
A.
, and
Castellanos
,
J. A.
,
2008
, “
Active Policy Learning for Robot Planning and Exploration Under Uncertainty
,”
Proceedings of Robotics: Science and Systems
,
Zurich, Switzerland
,
June 25–28
, pp.
321
328
.
20.
Lizotte
,
D. J.
,
Wang
,
T.
,
Bowling
,
M. H.
, and
Schuurmans
,
D.
,
2007
, “
Automatic Gait Optimization With Gaussian Process Regression
,”
Proceedings of IJCAI
,
Hyderabad, India
,
Jan. 6–12
, pp.
944
949
.
21.
Hutter
,
F.
,
Hoos
,
H. H.
, and
Leyton-Brown
,
K.
,
2011
, “
Sequential Model-Based Optimization for General Algorithm Configuration
,”
Proceedings of International Conference on Learning and Intelligent Optimization
,
Rome, Italy
,
Jan. 17–21
, pp.
507
523
.
22.
Bergstra
,
J. S.
,
Bardenet
,
R.
,
Bengio
,
Y.
, and
Kégl
,
B.
,
2011
, “
Algorithms for Hyper-Parameter Optimization
,”
Proceedings of Advances in Neural Information Processing Systems
,
Sierra Nevada, Spain
,
Dec. 16–17
, pp.
2546
2554
.
23.
Tran
,
A.
,
Sun
,
J.
,
Furlan
,
J. M.
,
Pagalthivarthi
,
K. V.
,
Visintainer
,
R. J.
, and
Wang
,
Y.
,
2019
, “
pBO-2GP-3B: A Batch Parallel Known/Unknown Constrained Bayesian Optimization With Feasibility Classification and Its Applications in Computational Fluid Dynamics
,”
Comput. Methods Appl. Mech. Eng.
,
347
, pp.
827
852
. 10.1016/j.cma.2018.12.033
24.
Tran
,
A.
,
Tran
,
M.
, and
Wang
,
Y.
,
2019
, “
Constrained Mixed-Integer Gaussian Mixture Bayesian Optimization and Its Applications in Designing Fractal and Auxetic Metamaterials
,”
Struct. Multidiscipl. Optim.
,
59
(
6
), pp.
2131
2154
. 10.1007/s00158-018-2182-1
25.
Shan
,
S.
, and
Wang
,
G. G.
,
2004
, “
An Efficient Pareto Set Identification Approach for Multiobjective Optimization on Black-Box Functions
,”
ASME J. Mech. Des.
,
127
(
5
), pp.
866
874
. 10.1115/1.1904639
26.
Shu
,
L.
,
Jiang
,
P.
,
Zhou
,
Q.
,
Shao
,
X.
,
Hu
,
J.
, and
Meng
,
X.
,
2018
, “
An On-Line Variable Fidelity Metamodel Assisted Multi-Objective Genetic Algorithm for Engineering Design Optimization
,”
Appl. Soft Comput.
,
66
, pp.
438
448
. 10.1016/j.asoc.2018.02.033
27.
Deb
,
K.
,
Pratap
,
A.
,
Agarwal
,
S.
, and
Meyarivan
,
T.
,
2002
, “
A Fast and Elitist Multiobjective Genetic Algorithm: NSGA-II
,”
IEEE Trans. Evol. Comput.
,
6
(
2
), pp.
182
197
.
28.
Ak
,
R.
,
Li
,
Y.
,
Vitelli
,
V.
,
Zio
,
E.
,
López Droguett
,
E.
, and
Magno Couto Jacinto
,
C.
,
2013
, “
NSGA-II-Trained Neural Network Approach to the Estimation of Prediction Intervals of Scale Deposition Rate in Oil & Gas Equipment
,”
Exp. Syst. Appl.
,
40
(
4
), pp.
1205
1212
. 10.1016/j.eswa.2012.08.018
29.
Shu
,
L.
,
Jiang
,
P.
,
Zhou
,
Q.
, and
Xie
,
T.
,
2019
, “
An Online Variable-Fidelity Optimization Approach for Multi-Objective Design Optimization
,”
Struct. Multidiscipl. Optim.
,
60
(
3
), pp.
1059
1077
. 10.1007/s00158-019-02256-0
30.
Sun
,
X.
,
Gong
,
D.
,
Jin
,
Y.
, and
Chen
,
S.
,
2013
, “
A New Surrogate-Assisted Interactive Genetic Algorithm With Weighted Semisupervised Learning
,”
IEEE Trans. Cybern.
,
43
(
2
), pp.
685
698
.
31.
Cheng
,
R.
,
Jin
,
Y.
,
Narukawa
,
K.
, and
Sendhoff
,
B.
,
2015
, “
A Multiobjective Evolutionary Algorithm Using Gaussian Process-Based Inverse Modeling
,”
IEEE Trans. Evol. Comput.
,
19
(
6
), pp.
838
856
.
32.
Li
,
M.
,
2011
, “
An Improved Kriging-Assisted Multi-Objective Genetic Algorithm
,”
ASME J. Mech. Des.
,
133
(
7
), p.
071008
.
33.
Sun
,
C.
,
Jin
,
Y.
,
Cheng
,
R.
,
Ding
,
J.
, and
Zeng
,
J.
,
2017
, “
Surrogate-Assisted Cooperative Swarm Optimization of High-Dimensional Expensive Problems
,”
IEEE Trans. Evol. Comput.
,
21
(
4
), pp.
644
660
. 10.1109/TEVC.2017.2675628
34.
Chen
,
G.
,
Han
,
X.
,
Liu
,
G.
,
Jiang
,
C.
, and
Zhao
,
Z.
,
2012
, “
An Efficient Multi-Objective Optimization Method for Black-Box Functions Using Sequential Approximate Technique
,”
Appl. Soft Comput.
,
12
(
1
), pp.
14
27
. 10.1016/j.asoc.2011.09.011
35.
Regis
,
R. G.
,
2014
, “
Evolutionary Programming for High-Dimensional Constrained Expensive Black-Box Optimization Using Radial Basis Functions
,”
IEEE Trans. Evol. Comput.
,
18
(
3
), pp.
326
347
.
36.
Datta
,
R.
, and
Regis
,
R. G.
,
2016
, “
A Surrogate-Assisted Evolution Strategy for Constrained Multi-Objective Optimization
,”
Exp. Syst. Appl.
,
57
, pp.
270
284
. 10.1016/j.eswa.2016.03.044
37.
An
,
H.
,
Chen
,
S.
, and
Huang
,
H.
,
2018
, “
Multi-Objective Optimization of a Composite Stiffened Panel for Hybrid Design of Stiffener Layout and Laminate Stacking Sequence
,”
Struct. Multidiscipl. Optim.
,
57
(
4
), pp.
1411
1426
. 10.1007/s00158-018-1918-2
38.
Knowles
,
J.
,
2006
, “
ParEGO: A Hybrid Algorithm With On-Line Landscape Approximation for Expensive Multiobjective Optimization Problems
,”
IEEE Trans. Evol. Comput.
,
10
(
1
), pp.
50
66
. 10.1109/TEVC.2005.851274
39.
Jeong
,
S.
, and
Obayashi
,
S.
,
2005
, “
Efficient Global Optimization (EGO) for Multi-Objective Problem and Data Mining
,”
Proceedings of 2005 IEEE Congress on Evolutionary Computation
,
Edinburgh, Scotland, UK
,
Sept. 2–5
,
IEEE
, pp.
2138
2145
.
40.
Ponweiser
,
W.
,
Wagner
,
T.
,
Biermann
,
D.
, and
Vincze
,
M.
,
2008
, “
Multiobjective Optimization on a Limited Budget of Evaluations Using Model-Assisted $ S $-Metric Selection
,”
Proceedings of International Conference on Parallel Problem Solving From Nature
,
Dortmund, Germany
,
Sept. 13–17
,
Springer
, pp.
784
794
.
41.
Gaudrie
,
D.
,
Riche
,
R. L.
,
Picheny
,
V.
,
Enaux
,
B.
, and
Herbert
,
V.
,
2018
, “
Budgeted Multi-Objective Optimization With a Focus on the Central Part of the Pareto Front-Extended Version
,” .
42.
Keane
,
A. J.
,
2006
, “
Statistical Improvement Criteria for Use in Multiobjective Design Optimization
,”
AIAA J.
,
44
(
4
), pp.
879
891
. 10.2514/1.16875
43.
Feliot
,
P.
,
Bect
,
J.
, and
Vazquez
,
E.
,
2017
, “
A Bayesian Approach to Constrained Single- and Multi-Objective Optimization
,”
J. Global Optim.
,
67
(
1
), pp.
97
133
. 10.1007/s10898-016-0427-3
44.
Forrester
,
A.
,
Sobester
,
A.
, and
Keane
,
A.
,
2008
,
Engineering Design via Surrogate Modelling: A Practical Guide
,
John Wiley & Sons
,
Hoboken, NJ
.
45.
Bautista
,
D. C. T.
,
2009
,
A Sequential Design for Approximating the Pareto Front Using the Expected Pareto Improvement Function
,
The Ohio State University
,
Columbus, OH
.
46.
Pandita
,
P.
,
Bilionis
,
I.
,
Panchal
,
J.
,
Gautham
,
B.
,
Joshi
,
A.
, and
Zagade
,
P.
,
2018
, “
Stochastic Multiobjective Optimization on a Budget: Application to Multipass Wire Drawing With Quantified Uncertainties
,”
Int. J. Uncert. Quant.
,
8
(
3
), pp.
233
249
.
47.
Calandra
,
R.
,
Peters
,
J.
, and
Deisenrothy
,
M.
,
2014
, “
Pareto Front Modeling for Sensitivity Analysis in Multi-Objective Bayesian Optimization
,”
Proceedings of NIPS Workshop on Bayesian Optimization
,
Montreal, QC, Canada
,
Dec. 8–13
.
48.
Zhan
,
D.
,
Cheng
,
Y.
, and
Liu
,
J.
,
2017
, “
Expected Improvement Matrix-Based Infill Criteria for Expensive Multiobjective Optimization
,”
IEEE Trans. Evol. Comput.
,
21
(
6
), pp.
956
975
. 10.1109/TEVC.2017.2697503
49.
Wu
,
J.
, and
Azarm
,
S.
,
2001
, “
Metrics for Quality Assessment of a Multiobjective Design Optimization Solution set
,”
ASME J. Mech. Des.
,
123
(
1
), pp.
18
25
. 10.1115/1.1329875
50.
Cheng
,
S.
,
Zhou
,
J.
, and
Li
,
M.
,
2015
, “
A New Hybrid Algorithm for Multi-Objective Robust Optimization With Interval Uncertainty
,”
ASME J. Mech. Des.
,
137
(
2
), p.
021401
. 10.1115/1.4029026
51.
Jin
,
R.
,
Chen
,
W.
, and
Sudjianto
,
A.
,
2002
, “
On Sequential Sampling for Global Metamodeling in Engineering Design
,”
Proceedings of ASME 2002 International Design Engineering Technical Conferences and Computers and Information in Engineering Conference
,
Montreal, Quebec, Canada
,
Sept. 29–Oct. 2
,
American Society of Mechanical Engineers
, pp.
539
548
.
52.
Shu
,
L.
,
Jiang
,
P.
,
Wan
,
L.
,
Zhou
,
Q.
,
Shao
,
X.
, and
Zhang
,
Y.
,
2017
, “
Metamodel-Based Design Optimization Employing a Novel Sequential Sampling Strategy
,”
Eng. Comput.
,
34
(
8
), pp.
2547
2564
. 10.1108/EC-01-2016-0034
53.
Rasmussen
,
C. E.
,
2004
, “Gaussian Processes in Machine Learning,”
Advanced Lectures on Machine Learning: ML Summer Schools 2003, Canberra, Australia, February 2–14, 2003, Tübingen, Germany, August 4–16, 2003, Revised Lectures
,
O.
Bousquet
,
U.
von Luxburg
, and
G.
Rätsch
, eds.,
Springer
,
Berlin, Heidelberg
, pp.
63
71
.
54.
Park
,
J.-S.
,
1994
, “
Optimal Latin-Hypercube Designs for Computer Experiments
,”
J. Stat. Plan. Inference
,
39
(
1
), pp.
95
111
. 10.1016/0378-3758(94)90115-5
55.
Wang
,
G. G.
,
2003
, “
Adaptive Response Surface Method Using Inherited Latin Hypercube Design Points
,”
ASME J. Mech. Des.
,
125
(
2
), pp.
210
220
. 10.1115/1.1561044
56.
Lophaven
,
S. N.
,
Nielsen
,
H. B.
, and
Søndergaard
,
J.
,
2002
, “
DACE-A Matlab Kriging Toolbox, Version 2.0
.”
57.
Deb
,
K.
,
Agrawal
,
S.
,
Pratap
,
A.
, and
Meyarivan
,
T.
,
2000
, “
A Fast Elitist Non-Dominated Sorting Genetic Algorithm for Multi-Objective Optimization: NSGA-II
,”
Proceedings of International Conference on Parallel Problem Solving From Nature
,
Paris, France
,
Sept. 18–20
,
Springer
, pp.
849
858
.
58.
Liu
,
Y.
, and
Collette
,
M.
,
2014
, “
Improving Surrogate-Assisted Variable Fidelity Multi-Objective Optimization Using a Clustering Algorithm
,”
Appl. Soft Comput.
,
24
, pp.
482
493
. 10.1016/j.asoc.2014.07.022
59.
Park
,
H.-S.
, and
Dang
,
X.-P.
,
2010
, “
Structural Optimization Based on CAD–CAE Integration and Metamodeling Techniques
,”
Comput. Aided Des.
,
42
(
10
), pp.
889
902
. 10.1016/j.cad.2010.06.003
60.
Kalyanmoy
,
D.
,
2001
,
Multi Objective Optimization Using Evolutionary Algorithms
,
John Wiley and Sons
,
Hoboken, NJ
.
61.
Coello
,
C. A. C.
,
2000
, “
Use of a Self-Adaptive Penalty Approach for Engineering Optimization Problems
,”
Comput. Ind.
,
41
(
2
), pp.
113
127
. 10.1016/S0166-3615(99)00046-9
You do not currently have access to this content.