Abstract

This work investigates surrogate modeling techniques for learning to approximate a computationally expensive function evaluation of 3D models. While in the past, 3D point clouds have been a data format that is too high dimensional for surrogate modeling, by leveraging advances in 3D object autoencoding neural networks, these point clouds can be mapped to a one-dimensional latent space. This leads to the fundamental research question: what surrogate modeling technique is most suitable for learning relationships between the 3D geometric features of the objects captured in the encoded latent vector and the physical phenomena captured in the evaluation software? Radial basis functions (RBFs), Kriging, and shallow 1D analogs of popular deep 2D image classification neural networks are investigated in this work. We find the nonintuitive result that departing from neural networks to decode latent representations of 3D objects into performance predictions is far more efficient than using a neural network decoder. In test cases using datasets of aircraft and watercraft 3D models, the non-neural network surrogate models achieve comparable accuracy to the neural network models. We find that an RBF surrogate model is able to approximate the lift and drag coefficients of 234 aircraft models with a mean absolute error of 1.97 × 10−3 and trains in only 3 seconds. Furthermore, the RBF surrogate model is able to rank a set of designs with an average percentile error of less than 8%. In comparison, a 1D ResNet achieves an average absolute error of 1.35 × 103 in 38 min for the same test case. We validate the comparable accuracy of the four techniques through a test case involving 214 3D watercraft models, but we also find that the distribution of the performance values of the data, in particular the presence of many outliers, has a significant negative impact on accuracy. These results contradict a common perception of neural networks as an efficient “one-size-fits-all” solution for learning black-box functions and suggests that even within systems that utilize multiple neural networks, potentially more efficient alternatives should be considered for each network in the system. Depending on the required accuracy of the application, this surrogate modeling approach could be used to approximate an expensive simulation software, or if the tolerance for error is low, it serves as a first pass which can narrow down the number of candidate designs to be analyzed more thoroughly.

References

References
1.
Naboni
,
E.
,
Zhang
,
Y.
,
Maccarini
,
A.
,
Hirsch
,
E.
, and
Lezzi
,
D.
,
2013
, “
Extending the Use of Parametric Simulation in Practice Through a Cloud Based Online Service
,”
IBPSA Italy-Conference of International Building Performance Simulation Association
,
Bozen, Italy
,
Jan. 30–Feb. 1
.
2.
Rößger
,
P.
, and
Richter
,
A.
,
2018
, “
Performance of Different Optimization Concepts for Reactive Flow Systems Based on Combined Cfd and Response Surface Methods
,”
Comput. Chem. Eng.
,
108
, pp.
232
239
. 10.1016/j.compchemeng.2017.09.008
3.
Wang
,
G. G.
, and
Shan
,
S.
,
2007
, “
Review of Metamodeling Techniques in Support of Engineering Design Optimization
,”
ASME J. Mech. Des.
,
129
(
4
), pp.
370
380
. 10.1115/1.2429697
4.
Hardee
,
E.
,
Chang
,
K.-H.
,
Tu
,
J.
,
Choi
,
K. K.
,
Grindeanu
,
I.
, and
Yu
,
X.
,
1999
, “
A CAD-Based Design Parameterization for Shape Optimization of Elastic Solids
,”
Adv. Eng. Softw.
,
30
(
3
), pp.
185
199
. 10.1016/S0965-9978(98)00065-9
5.
Samareh
,
J. A.
,
1999
, “
A Survey of Shape Parameterization Techniques
,”
CEAS/AIAA/ICASE/NASA Langley International Forum on Aeroelasticity and Structural Dynamics
,
Williamsburg, VA
,
June 22–25
.
6.
Sripawadkul
,
V.
,
Padulo
,
M.
, and
Guenov
,
M.
,
2010
, “
A comparison of airfoil shape parameterization techniques for early design optimization
,”
13th AIAA/ISSMO Multidisciplinary Analysis Optimization Conference
, p.
9050
.
7.
Shahrokhi
,
A.
, and
Jahangirian
,
A.
,
2007
, “
Airfoil Shape Parameterization for Optimum Navier–Stokes Design With Genetic Algorithm
,”
Aerosp. Sci. Technol.
,
11
(
6
), pp.
443
450
. 10.1016/j.ast.2007.04.004
8.
Chang
,
A. X.
,
Hanrahan
,
P.
, and
Savva
,
M.
,
2015
, “
Semantically-Enriched 3D Models for Common-sense Knowledge
,”
Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition
,
Boston, MA
,
June 8–10
.
9.
Wu
,
Z.
,
Song
,
S.
,
Khosla
,
A.
,
Yu
,
F.
,
Zhang
,
L.
,
Tang
,
X.
, and
Xiao
,
J.
,
2015
, “
3D Shapenets: A Deep Representation for Volumetric Shapes
,”
Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition
,
Boston, MA
,
June 7–12
, pp.
1912
1920
.
10.
Burnap
,
A.
,
Liu
,
Y.
,
Pan
,
Y.
,
Lee
,
H.
,
Gonzalez
,
R.
, and
Papalambros
,
P. Y.
,
2016
, “
Estimating and Exploring the Product Form Design Space Using Deep Generative Models
,”
Ann. Arbor
,
1001
, p.
48109
.
11.
Achlioptas
,
P.
,
Diamanti
,
O.
,
Mitliagkas
,
I.
, and
Guibas
,
L.
,
2017
, “
Representation Learning and Adversarial Generation of 3D Point Clouds
,” e-print arXiv:1707.02392.
12.
Qi
,
C. R.
,
Su
,
H.
,
Mo
,
K.
, and
Guibas
,
L. J.
,
2017
, “
Pointnet: Deep Learning on Point Sets for 3D Classification and Segmentation
,”
Proceedings of the Computer Vision and Pattern Recognition (CVPR)
,
Honolulu, HI
,
July 21–26
,
IEEE
,
New York
, Vol.
1
(
2
), p.
4
.
13.
Groueix
,
T.
,
Fisher
,
M.
,
Kim
,
V. G.
,
Russell
,
B. C.
, and
Aubry
,
M.
,
2018
, “
Atlasnet: A Papier-Mâché Approach to Learning 3D Surface Generation
,” e-print arXiv:1802.05384.
14.
Nash
,
C.
, and
Williams
,
C. K.
,
2017
, “
The Shape Variational Autoencoder: A Deep Generative Model of Part-Segmented 3D Objects
,”
Computer Graphics Forum
,
36
(
5
), pp.
1
12
.
15.
Maturana
,
D.
, and
Scherer
,
S.
,
2015
, “
Voxnet: A 3D Convolutional Neural Network for Real-time Object Recognition
,”
2015 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)
,
Hamburg, Germany
,
Sept. 28–Oct. 2
,
IEEE
,
New York
, pp.
922
928
.
16.
Riegler
,
G.
,
Osman Ulusoy
,
A.
, and
Geiger
,
A.
,
2017
, “
Octnet: Learning Deep 3D Representations at High Resolutions
,”
Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition
,
Honolulu, HI
,
July 21–26
, pp.
3577
3586
.
17.
Wang
,
D. Z.
, and
Posner
,
I.
,
2015
, “
Voting for Voting in Online Point Cloud Object Detection
,”
Robotics: Science and Systems
,
1
(
3
), pp.
10
15607
.
18.
Fang
,
Y.
,
Xie
,
J.
,
Dai
,
G.
,
Wang
,
M.
,
Zhu
,
F.
,
Xu
,
T.
, and
Wong
,
E.
,
2015
, “
3D Deep Shape Descriptor
,”
Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition
,
Boston, MA
,
June 8–10
, pp.
2319
2328
.
19.
Mullur
,
A. A.
, and
Messac
,
A.
,
2005
, “
Extended Radial Basis Functions: More Flexible and Effective Metamodeling
,”
AIAA J.
,
43
(
6
), pp.
1306
1315
. 10.2514/1.11292
20.
Jin
,
R.
,
Chen
,
W.
, and
Simpson
,
T. W.
,
2001
, “
Comparative Studies of Metamodelling Techniques Under Multiple Modelling Criteria
,”
Struct. Multidisciplinary Optim.
,
23
(
1
), pp.
1
13
. 10.1007/s00158-001-0160-4
21.
Ellbrant
,
L.
,
Eriksson
,
L.-E.
, and
Mårtensson
,
H.
,
2012
, “
CFD Optimization of a Transonic Compressor Using Multiobjective Ga and Metamodels
,”
Proceedings of the 28th International Congress of the Aeronautical Sciences
,
Brisbane, Australia
,
Sept. 23–28
, pp.
23
28
.
22.
Sacks
,
J.
,
Schiller
,
S. B.
, and
Welch
,
W. J.
,
1989
, “
Designs for Computer Experiments
,”
Technometrics
,
31
(
1
), pp.
41
47
. 10.1080/00401706.1989.10488474
23.
Kleijnen
,
J. P.
,
2009
, “
Kriging Metamodeling in Simulation: A Review
,”
Eur. J. Oper. Res.
,
192
(
3
), pp.
707
716
. 10.1016/j.ejor.2007.10.013
24.
Simpson
,
T. W.
,
Mauery
,
T. M.
,
Korte
,
J. J.
, and
Mistree
,
F.
,
2001
, “
Kriging Models for Global Approximation in Simulation-Based Multidisciplinary Design Optimization
,”
AIAA J.
,
39
(
12
), pp.
2233
2241
. 10.2514/2.1234
25.
Jia
,
G.
, and
Taflanidis
,
A. A.
,
2013
, “
Kriging Metamodeling for Approximation of High-Dimensional Wave and Surge Responses in Real-Time Storm/hurricane Risk Assessment
,”
Comput. Methods Appl. Mech. Eng.
,
261
, pp.
24
38
. 10.1016/j.cma.2013.03.012
26.
Bouhlel
,
M. A.
,
Bartoli
,
N.
,
Otsmane
,
A.
, and
Morlier
,
J.
,
2016
, “
Improving Kriging Surrogates of High-Dimensional Design Models by Partial Least Squares Dimension Reduction
,”
Struct. Multidisciplinary Optim.
,
53
(
5
), pp.
935
952
. 10.1007/s00158-015-1395-9
27.
Wold
,
H.
,
1975
, “
Soft Modelling by Latent Variables: the Non-linear Iterative Partial Least Squares (nipals) Approach
,”
J. Appl. Probab.
,
12
(
S1
), pp.
117
142
. 10.1017/S0021900200047604
28.
Caixeta
,
P.
, Jr.
, and
Marques
,
F.
,
2010
, “
Neural Network Metamodel-based MDO for Wing Design Considering Aeroelastic Constraints
,”
51st AIAA/ASME/ASCE/AHS/ASC Structures, Structural Dynamics, and Materials Conference, 18th AIAA/ASME/AHS Adaptive Structures Conference 12th
,
Orlando, FL
,
Apr. 12–15
, p.
2762
.
29.
Ferreiro-Cabello
,
J.
,
Fraile-Garcia
,
E.
,
de Pison Ascacibar
,
E. M.
, and
de Pison Ascacibar
,
F. J. M.
,
2018
, “
Metamodel-Based Design Optimization of Structural One-Way Slabs Based on Deep Learning Neural Networks to Reduce Environmental Impact
,”
Eng. Struct.
,
155
, pp.
91
101
. 10.1016/j.engstruct.2017.11.005
30.
Sreekanth
,
J.
, and
Datta
,
B.
,
2010
, “
Multi-Objective Management of Saltwater Intrusion in Coastal Aquifers Using Genetic Programming and Modular Neural Network Based Surrogate Models
,”
J. Hydrol.
,
393
(
3–4
), pp.
245
256
. 10.1016/j.jhydrol.2010.08.023
31.
Krizhevsky
,
A.
,
Sutskever
,
I.
, and
Hinton
,
G. E.
,
2012
, “
Imagenet Classification With Deep Convolutional Neural Networks
,”
Twenty-sixth Conference on Neural Information Processing Systems
,
Lake Tahoe, NV
,
Dec. 3–8
, pp.
1097
1105
.
32.
Howard
,
A. G.
,
2013
, “
Some Improvements on Deep Convolutional Neural Network Based Image Classification
,”
International Conference on Learning Representations 2013
,
Scottsdale, AZ
,
May 2–4
.
33.
Esteva
,
A.
,
Kuprel
,
B.
,
Novoa
,
R. A.
,
Ko
,
J.
,
Swetter
,
S. M.
,
Blau
,
H. M.
, and
Thrun
,
S.
,
2017
, “
Dermatologist-Level Classification of Skin Cancer With Deep Neural Networks
,”
Nature
,
542
(
7639
), p.
115
. 10.1038/nature21056
34.
Simonyan
,
K.
, and
Zisserman
,
A.
,
2014
, “
Very Deep Convolutional Networks for Large-scale Image Recognition
,” e-print arXiv:1409.1556.
35.
He
,
K.
,
Zhang
,
X.
,
Ren
,
S.
, and
Sun
,
J.
,
2016
, “
Deep Residual Learning for Image Recognition
,”
Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition
,
Las Vegas, NV
,
June 26–July 1
, pp.
770
778
.
36.
Xu
,
Y.
,
Du
,
J.
,
Dai
,
L.-R.
, and
Lee
,
C.-H.
,
2015
, “
A Regression Approach to Speech Enhancement Based on Deep Neural Networks
,”
IEEE/ACM Trans. Audio Speech Language Process.
,
23
(
1
), pp.
7
19
. 10.1109/TASLP.2014.2364452
37.
Xu
,
Y.
,
Du
,
J.
,
Dai
,
L.-R.
, and
Lee
,
C.-H.
,
2014
, “
An Experimental Study on Speech Enhancement Based on Deep Neural Networks
,”
IEEE Signal Process. Lett.
,
21
(
1
), pp.
65
68
. 10.1109/LSP.2013.2291240
38.
Toshev
,
A.
, and
Szegedy
,
C.
,
2014
, “
Deeppose: Human Pose Estimation via Deep Neural Networks
,”
Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition
,
Columbus, OH
,
June 24–27
, pp.
1653
1660
.
39.
Held
,
D.
,
Thrun
,
S.
, and
Savarese
,
S.
,
2016
, “
Learning to Track at 100 fps With Deep Regression Networks
,”
European Conference on Computer Vision
,
Amsterdam, Netherlands
,
Oct. 8–16
, pp.
749
765
.
40.
Kossaifi
,
J.
,
Lipton
,
Z. C.
,
Khanna
,
A.
,
Furlanello
,
T.
, and
Anandkumar
,
A.
,
2017
, “
Tensor Regression Networks
,” e-print arXiv:1707.08308,
2
(
4
).
41.
Solovyev
,
R. A.
,
Vakhrushev
,
M.
,
Radionov
,
A.
,
Aliev
,
V.
, and
Shvets
,
A. A.
,
2018
, “
Deep Learning Approaches for Understanding Simple Speech Commands
,” e-print arXiv:1810.02364.
42.
Hurtado
,
D. M.
,
Uziela
,
K.
, and
Elofsson
,
A.
,
2018
, “
Deep Transfer Learning in the Assessment of the Quality of Protein Models
,” e-print arXiv:1804.06281.
43.
Bouhlel
,
M. A.
,
2018
, “
Surrogate Modeling Toolbox
,” https://github.com/SMTorg/smt.
44.
Ioffe
,
S.
, and
Szegedy
,
C.
,
2015
, “
Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift
,”
International Conference on Machine Learning
,
Lille, France
,
July 6–July 11
, pp.
448
456
.
45.
Jasak
,
H.
,
Jemcov
,
A.
, and
Tukovic
,
Z.
,
2007
, “
Openfoam: A C++ Library for Complex Physics Simulations
,”
International Workshop on Coupled Methods in Numerical Dynamics
,
Dubrovnik, Croatia
,
Sept. 19–21
, pp.
1
20
.
46.
Jasak
,
H.
,
2009
, “
Openfoam: Open Source CFD in Research and Industry
,”
Int. J. Naval Arch. Ocean Eng.
,
1
(
2
), pp.
89
94
.
47.
Chen
,
G.
,
Xiong
,
Q.
,
Morris
,
P. J.
,
Paterson
,
E. G.
,
Sergeev
,
A.
, and
Wang
,
Y.
,
2014
, “
Openfoam for Computational Fluid Dynamics
,”
Not. AMS
,
61
(
4
), pp.
354
363
.
48.
Higuera
,
P.
,
Lara
,
J. L.
, and
Losada
,
I. J.
,
2013
, “
Simulating Coastal Engineering Processes With Openfoam®
,”
Coastal Eng.
,
71
, pp.
119
134
. 10.1016/j.coastaleng.2012.06.002
49.
Higuera
,
P.
,
Lara
,
J. L.
, and
Losada
,
I. J.
,
2013
, “
Realistic Wave Generation and Active Wave Absorption for Navier–Stokes Models: Application to Openfoam®
,”
Coastal Eng.
,
71
, pp.
102
118
. 10.1016/j.coastaleng.2012.07.002
50.
Kunkelmann
,
C.
, and
Stephan
,
P.
,
2009
, “
CFD Simulation of Boiling Flows Using the Volume-of-Fluid Method Within Openfoam
,”
Numer. Heat Transfer, Part A: Appl.
,
56
(
8
), pp.
631
646
. 10.1080/10407780903423908
51.
Petit
,
O.
,
Page
,
M.
,
Beaudoin
,
M.
, and
Nilsson
,
H.
,
2009
, “
The Ercoftac Centrifugal Pump Openfoam Case-study
,”
Proceedings of the Third IAHR International Meeting of the Workgroup on Cavitation and Dynamic Problem in Hydraulic Machinery and Systems
,
Brno, Czech Republic
,
Oct. 14–16
.
52.
Lysenko
,
D. A.
,
Ertesvåg
,
I. S.
, and
Rian
,
K. E.
,
2013
, “
Modeling of Turbulent Separated Flows Using Openfoam
,”
Comput. Fluids
,
80
, pp.
408
422
. 10.1016/j.compfluid.2012.01.015
53.
Kassem
,
H. I.
,
Saqr
,
K. M.
,
Aly
,
H. S.
,
Sies
,
M. M.
, and
Wahid
,
M. A.
,
2011
, “
Implementation of the Eddy Dissipation Model of Turbulent Non-Premixed Combustion in Openfoam
,”
Int. Commun. Heat Mass Transfer
,
38
(
3
), pp.
363
367
. 10.1016/j.icheatmasstransfer.2010.12.012
54.
Lysenko
,
D. A.
,
Ertesvåg
,
I. S.
, and
Rian
,
K. E.
,
2012
, “
Large-Eddy Simulation of the Flow Over a Circular Cylinder At Reynolds Number 3900 Using the Openfoam Toolbox
,”
Flow Turbulence Combust.
,
89
(
4
), pp.
491
518
. 10.1007/s10494-012-9405-0
55.
Iturrioz
,
A.
,
Guanche
,
R.
,
Lara
,
J.
,
Vidal
,
C.
, and
Losada
,
I.
,
2015
, “
Validation of Openfoam® for Oscillating Water Column Three-Dimensional Modeling
,”
Ocean Eng.
,
107
, pp.
222
236
. 10.1016/j.oceaneng.2015.07.051
56.
Silva
,
L. F. L.
, and
Lage
,
P. L.
,
2011
, “
Development and Implementation of a Polydispersed Multiphase Flow Model in Openfoam
,”
Comput. Chem. Eng.
,
35
(
12
), pp.
2653
2666
. 10.1016/j.compchemeng.2011.04.011
You do not currently have access to this content.