Abstract

We introduce a novel method for Gaussian process (GP) modeling of massive datasets called globally approximate Gaussian process (GAGP). Unlike most large-scale supervised learners such as neural networks and trees, GAGP is easy to fit and can interpret the model behavior, making it particularly useful in engineering design with big data. The key idea of GAGP is to build a collection of independent GPs that use the same hyperparameters but randomly distribute the entire training dataset among themselves. This is based on our observation that the GP hyperparameter approximations change negligibly as the size of the training data exceeds a certain level, which can be estimated systematically. For inference, the predictions from all GPs in the collection are pooled, allowing the entire training dataset to be efficiently exploited for prediction. Through analytical examples, we demonstrate that GAGP achieves very high predictive power matching (and in some cases exceeding) that of state-of-the-art supervised learning methods. We illustrate the application of GAGP in engineering design with a problem on data-driven metamaterials, using it to link reduced-dimension geometrical descriptors of unit cells and their properties. Searching for new unit cell designs with desired properties is then achieved by employing GAGP in inverse optimization.

References

References
1.
Yan
,
W.
,
Lin
,
S.
,
Kafka
,
O. L.
,
Lian
,
Y.
,
Yu
,
C.
,
Liu
,
Z.
,
Yan
,
J.
,
Wolff
,
S.
,
Wu
,
H.
, and
Ndip-Agbor
,
E.
,
2018
, “
Data-Driven Multi-Scale Multi-Physics Models to Derive Process–Structure–Property Relationships for Additive Manufacturing
,”
Comput. Mech.
,
61
(
5
), pp.
521
541
. 10.1007/s00466-018-1539-z
2.
Mozaffar
,
M.
,
Paul
,
A.
,
Al-Bahrani
,
R.
,
Wolff
,
S.
,
Choudhary
,
A.
,
Agrawal
,
A.
,
Ehmann
,
K.
, and
Cao
,
J.
,
2018
, “
Data-Driven Prediction of the High-Dimensional Thermal History in Directed Energy Deposition Processes via Recurrent Neural Networks
,”
Manuf. Lett.
,
18
, pp.
35
39
. 10.1016/j.mfglet.2018.10.002
3.
Ghumman
,
U. F.
,
Iyer
,
A.
,
Dulal
,
R.
,
Munshi
,
J.
,
Wang
,
A.
,
Chien
,
T.
,
Balasubramanian
,
G.
, and
Chen
,
W.
,
2018
, “
A Spectral Density Function Approach for Active Layer Design of Organic Photovoltaic Cells
,”
ASME J. Mech. Des.
,
140
(
11
), p.
111408
. 10.1115/1.4040912
4.
Bessa
,
M. A.
,
Bostanabad
,
R.
,
Liu
,
Z.
,
Hu
,
A.
,
Apley
,
D. W.
,
Brinson
,
C.
,
Chen
,
W.
, and
Liu
,
W. K.
,
2017
, “
A Framework for Data-Driven Analysis of Materials Under Uncertainty: Countering the Curse of Dimensionality
,”
Comput. Method Appl. M.
,
320
, pp.
633
667
. 10.1016/j.cma.2017.03.037
5.
Bostanabad
,
R.
,
Chen
,
W.
, and
Apley
,
D. W.
,
2016
, “
Characterization and Reconstruction of 3D Stochastic Microstructures via Supervised Learning
,”
J. Microsc.
,
264
(
3
), pp.
282
297
. 10.1111/jmi.12441
6.
Bostanabad
,
R.
,
Zhang
,
Y.
,
Li
,
X.
,
Kearney
,
T.
,
Brinson
,
L. C.
,
Apley
,
D. W.
,
Liu
,
W. K.
, and
Chen
,
W.
,
2018
, “
Computational Microstructure Characterization and Reconstruction: Review of the State-of-the-Art Techniques
,”
Prog. Mater. Sci.
,
95
, pp.
1
41
. 10.1016/j.pmatsci.2018.01.005
7.
Li
,
X.
,
Yang
,
Z.
,
Brinson
,
L. C.
,
Choudhary
,
A.
,
Agrawal
,
A.
, and
Chen
,
W.
,
2018
, “
A Deep Adversarial Learning Methodology for Designing Microstructural Material Systems
,”
ASME 2018 International Design Engineering Technical Conferences and Computers and Information in Engineering Conference
,
Quebec City, Quebec, Canada
,
Aug. 26–29
.
8.
Bostanabad
,
R.
,
Bui
,
A. T.
,
Xie
,
W.
,
Apley
,
D. W.
, and
Chen
,
W.
,
2016
, “
Stochastic Microstructure Characterization and Reconstruction via Supervised Learning
,”
Acta Mater.
,
103
, pp.
89
102
. 10.1016/j.actamat.2015.09.044
9.
Schumacher
,
C.
,
Bickel
,
B.
,
Rys
,
J.
,
Marschner
,
S.
,
Daraio
,
C.
, and
Gross
,
M.
,
2015
, “
Microstructures to Control Elasticity in 3D Printing
,”
ACM Trans. Graph.
,
34
(
4
), p.
136
. 10.1145/2766926
10.
Panetta
,
J.
,
Zhou
,
Q.
,
Malomo
,
L.
,
Pietroni
,
N.
,
Cignoni
,
P.
, and
Zorin
,
D.
,
2015
, “
Elastic Textures for Additive Fabrication
,”
ACM Trans. Graph.
,
34
(
4
), p.
135
. 10.1145/2766937
11.
Zhu
,
B.
,
Skouras
,
M.
,
Chen
,
D.
, and
Matusik
,
W.
,
2017
, “
Two-Scale Topology Optimization With Microstructures
,”
ACM Trans. Graph.
,
36
(
5
), p.
164
.10.1145/3072959.3126835
12.
Chen
,
D.
,
Skouras
,
M.
,
Zhu
,
B.
, and
Matusik
,
W.
,
2018
, “
Computational Discovery of Extremal Microstructure Families
,”
Sci. Adv.
,
4
(
1
), p.
eaao7005
. 10.1126/sciadv.aao7005
13.
LeCun
,
Y.
,
Bengio
,
Y.
, and
Hinton
,
G.
,
2015
, “
Deep Learning
,”
Nature
,
521
(
7553
), pp.
436
444
. 10.1038/nature14539
14.
Chen
,
T.
, and
Guestrin
,
C.
, “
Xgboost: A Scalable Tree Boosting System
,”
2016
,
Proceedings of 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining
,
San Francisco, CA
,
Aug. 13–17
, pp.
785
794
.
15.
Hassaninia
,
I.
,
Bostanabad
,
R.
,
Chen
,
W.
, and
Mohseni
,
H.
,
2017
, “
Characterization of the Optical Properties of Turbid Media by Supervised Learning of Scattering Patterns
,”
Sci. Rep.
,
7
(
1
), p.
15259
. 10.1038/s41598-017-15601-4
16.
Tao
,
S.
,
Shintani
,
K.
,
Bostanabad
,
R.
,
Chan
,
Y.-C.
,
Yang
,
G.
,
Meingast
,
H.
, and
Chen
,
W.
,
2017
, “
Enhanced Gaussian Process Metamodeling and Collaborative Optimization for Vehicle Suspension Design Optimization
,”
ASME 2017 International Design Engineering Technical Conferences and Computers and Information in Engineering Conference
,
Cleveland, OH
,
Aug. 6–9
, Paper No. DETC2017-67976.
17.
Bostanabad
,
R.
,
Liang
,
B.
,
Gao
,
J.
,
Liu
,
W. K.
,
Cao
,
J.
,
Zeng
,
D.
,
Su
,
X.
,
Xu
,
H.
,
Li
,
Y.
, and
Chen
,
W.
,
2018
, “
Uncertainty Quantification in Multiscale Simulation of Woven Fiber Composites
,”
Comput. Method Appl. M.
,
338
, pp.
506
532
. 10.1016/j.cma.2018.04.024
18.
Zhang
,
W.
,
Bostanabad
,
R.
,
Liang
,
B.
,
Su
,
X.
,
Zeng
,
D.
,
Bessa
,
M. A.
,
Wang
,
Y.
,
Chen
,
W.
, and
Cao
,
J.
,
2019
, “
A Numerical Bayesian-Calibrated Characterization Method for Multiscale Prepreg Preforming Simulations With Tension-Shear Coupling
,”
Compos. Sci. Technol.
,
170
, pp.
15
24
. 10.1016/j.compscitech.2018.11.019
19.
Gramacy
,
R. B.
, and
Lee
,
H. K. H.
,
2008
, “
Bayesian Treed Gaussian Process Models With an Application to Computer Modeling
,”
J. Am. Stat. Assoc.
,
103
(
483
), pp.
1119
1130
. 10.1198/016214508000000689
20.
Kim
,
H.-M.
,
Mallick
,
B. K.
, and
Holmes
,
C.
,
2005
, “
Analyzing Nonstationary Spatial Data Using Piecewise Gaussian Processes
,”
J. Am. Stat. Assoc.
,
100
(
470
), pp.
653
668
. 10.1198/016214504000002014
21.
Rasmussen
,
C. E.
,
2006
,
Gaussian Processes for Machine Learning
,
MIT Press
,
Cambridge, MA
.
22.
Tresp
,
V.
,
2000
, “
A Bayesian Committee Machine
,”
Neural Comput.
,
12
(
11
), pp.
2719
2741
. 10.1162/089976600300014908
23.
Herbrich
,
R.
,
Lawrence
,
N. D.
, and
Seeger
,
M.
,
2003
, “
Fast Sparse Gaussian Process Methods: The Informative Vector Machine
,”
Proc. Advances in Neural Information Processing Systems
, pp.
625
632
.
24.
Seeger
,
M.
,
Williams
,
C.
, and
Lawrence
,
N.
,
2003
, “
Fast Forward Selection to Speed Up Sparse Gaussian Process Regression
,”
Proceedings of Artificial Intelligence and Statistics 9
.
25.
Williams
,
C. K.
, and
Seeger
,
M.
,
2001
, “
Using the Nyström Method to Speed Up Kernel Machines
,”
Proceedings of Advances in Neural Information Processing Systems
, pp.
682
688
.
26.
Rasmussen
,
C. E.
, and
Quinonero-Candela
,
J.
,
2005
, “
Healing the Relevance Vector Machine Through Augmentation
,”
Proceedings of 22nd International Conference on Machine Learning
,
Aug. 7–11
, pp.
689
696
.
27.
Wahba
,
G.
,
1990
,
Spline Models for Observational Data
, Vol.
59
, Siam.
28.
Gramacy
,
R. B.
, and
Apley
,
D. W.
,
2015
, “
Local Gaussian Process Approximation for Large Computer Experiments
,”
J. Comput. Graph. Stat.
,
24
(
2
), pp.
561
578
. 10.1080/10618600.2014.914442
29.
Garcia
,
D. J.
,
Mozaffar
,
M.
,
Ren
,
H. Q.
,
Correa
,
J. E.
,
Ehmann
,
K.
,
Cao
,
J.
, and
You
,
F. Q.
,
2019
, “
Sustainable Manufacturing With Cyber-Physical Discrete Manufacturing Networks: Overview and Modeling Framework
,”
ASME J. Manuf. Sci. Eng.
,
141
(
2
), p.
021013
. 10.1115/1.4041833
30.
Mozaffar
,
M.
,
Ndip-Agbor
,
E.
,
Stephen
,
L.
,
Wagner
,
G. J.
,
Ehmann
,
K.
, and
Cao
,
J.
,
2019
, “
Acceleration Strategies for Explicit Finite Element Analysis of Metal Powder-Based Additive Manufacturing Processes Using Graphical Processing Units
,”
Comput. Mech.
, pp.
1
16
. 10.1007/s00466-019-01685-4
31.
Han
,
Y.
, and
Lu
,
W. F.
,
2018
, “
A Novel Design Method for Nonuniform Lattice Structures Based on Topology Optimization
,”
ASME J. Mech. Des.
,
140
(
9
), p.
091403
. 10.1115/1.4040546
32.
Li
,
D.
,
Dai
,
N.
,
Tang
,
Y.
,
Dong
,
G.
, and
Zhao
,
Y. F.
,
2019
, “
Design and Optimization of Graded Cellular Structures With Triply Periodic Level Surface-Based Topological Shapes
,”
ASME J. Mech. Des.
,
141
(
7
), p.
071402
.10.1115/1.4042617
33.
Du
,
Z.
,
Zhou
,
X.-Y.
,
Picelli
,
R.
, and
Kim
,
H. A.
,
2018
, “
Connecting Microstructures for Multiscale Topology Optimization With Connectivity Index Constraints
,”
ASME J. Mech. Des.
,
140
(
11
), p.
111417
. 10.1115/1.4041176
34.
Fu
,
J.
,
Xia
,
L.
,
Gao
,
L.
,
Xiao
,
M.
, and
Li
,
H.
,
2019
, “
Topology Optimization of Periodic Structures With Substructuring
,”
ASME J. Mech. Des.
,
141
(
7
), p.
071403
. 10.1115/1.4042616
35.
Bostanabad
,
R.
,
Kearney
,
T.
,
Tao
,
S.
,
Apley
,
D. W.
, and
Chen
,
W.
,
2018
, “
Leveraging the Nugget Parameter for Efficient Gaussian Process Modeling
,”
Int. J. Numer Meth. Eng.
,
114
(
5
), pp.
501
516
. 10.1002/nme.5751
36.
MacDonald
,
B.
,
Ranjan
,
P.
, and
Chipman
,
H.
,
2015
, “
GPfit: AnRPackage for Fitting a Gaussian Process Model to Deterministic Simulator Outputs
,”
J. Stat. Software
,
64
, p.
12
. 10.18637/jss.v064.i12
37.
Plumlee
,
M.
, and
Apley
,
D. W.
,
2017
, “
Lifted Brownian Kriging Models
,”
Technometrics
,
59
(
2
), pp.
165
177
. 10.1080/00401706.2016.1211555
38.
Ranjan
,
P.
,
Haynes
,
R.
, and
Karsten
,
R.
,
2011
, “
A Computationally Stable Approach to Gaussian Process Interpolation of Deterministic Computer Simulation Data
,”
Technometrics
,
53
(
4
), pp.
366
378
. 10.1198/TECH.2011.09141
39.
Sacks
,
J.
,
Schiller
,
S. B.
, and
Welch
,
W. J.
,
1989
, “
Designs for Computer Experiments
,”
Technometrics
,
31
(
1
), pp.
41
47
. 10.1080/00401706.1989.10488474
40.
Toal
,
D. J. J.
,
Bressloff
,
N. W.
, and
Keane
,
A. J.
,
2008
, “
Kriging Hyperparameter Tuning Strategies
,”
AIAA J.
,
46
(
5
), pp.
1240
1252
. 10.2514/1.34822
41.
Audet
,
C.
, and
Dennis
,
J. E.
, Jr
,
2002
, “
Analysis of Generalized Pattern Searches
,”
SIAM J. Optim.
,
13
(
3
), pp.
889
903
. 10.1137/S1052623400378742
42.
Zhao
,
L.
,
Choi
,
K.
, and
Lee
,
I.
,
2011
, “
Metamodeling Method Using Dynamic Kriging for Design Optimization
,”
AIAA J.
,
49
(
9
), pp.
2034
2046
. 10.2514/1.J051017
43.
Toal
,
D. J.
,
Bressloff
,
N.
,
Keane
,
A.
, and
Holden
,
C.
,
2011
, “
The Development of a Hybridized Particle Swarm for Kriging Hyperparameter Tuning
,”
Eng. Optim.
,
43
(
6
), pp.
675
699
. 10.1080/0305215X.2010.508524
44.
Conti
,
S.
,
Gosling
,
J. P.
,
Oakley
,
J. E.
, and
O'Hagan
,
A.
,
2009
, “
Gaussian Process Emulation of Dynamic Computer Codes
,”
Biometrika
,
96
(
3
), pp.
663
676
. 10.1093/biomet/asp028
45.
Arendt
,
P. D.
,
Apley
,
D. W.
, and
Chen
,
W.
,
2012
, “
Quantification of Model Uncertainty: Calibration, Model Discrepancy, and Identifiability
,”
ASME J. Mech. Des.
,
134
(
10
), p.
100908
. 10.1115/1.4007390
46.
Arendt
,
P. D.
,
Apley
,
D. W.
,
Chen
,
W.
,
Lamb
,
D.
, and
Gorsich
,
D.
,
2012
, “
Improving Identifiability in Model Calibration Using Multiple Responses
,”
ASME J. Mech. Des.
,
134
(
10
), p.
100909
. 10.1115/1.4007573
47.
Bayarri
,
M.
,
Berger
,
J.
,
Cafeo
,
J.
,
Garcia-Donato
,
G.
,
Liu
,
F.
,
Palomo
,
J.
,
Parthasarathy
,
R.
,
Paulo
,
R.
,
Sacks
,
J.
, and
Walsh
,
D.
,
2007
, “
Computer Model Validation With Functional Output
,”
Ann. Stat.
,
35
(
5
), pp.
1874
1906
. 10.1214/009053607000000163
48.
Conti
,
S.
, and
O’Hagan
,
A.
,
2010
, “
Bayesian Emulation of Complex Multi-Output and Dynamic Computer Models
,”
J. Stat. Plann. Inference
,
140
(
3
), pp.
640
651
. 10.1016/j.jspi.2009.08.006
49.
Sobol
,
I. M.
,
1967
, “
On the Distribution of Points in a Cube and the Approximate Evaluation of Integrals
,”
Zh. Vychisl. Mat. Mat. Fiz.
,
7
(
4
), pp.
784
802
.
50.
Sobol
,
I. M.
,
1998
, “
On Quasi-Monte Carlo Integrations
,”
Math. Comput. Simulat.
,
47
(
2–5
), pp.
103
112
. 10.1016/S0378-4754(98)00096-2
51.
Hastie
,
T.
,
Tibshirani
,
R.
,
Friedman
,
J.
,
Hastie
,
T.
,
Friedman
,
J.
, and
Tibshirani
,
R.
,
2009
,
The Elements of Statistical Learning
,
Springer
,
New York
.
52.
Bastos
,
L. S.
, and
O’Hagan
,
A.
,
2009
, “
Diagnostics for Gaussian Process Emulators
,”
Technometrics
,
51
(
4
), pp.
425
438
. 10.1198/TECH.2009.08019
53.
Ben-Ari
,
E. N.
, and
Steinberg
,
D. M.
,
2007
, “
Modeling Data From Computer Experiments: An Empirical Comparison of Kriging With MARS and Projection Pursuit Regression
,”
Qual. Eng.
,
19
(
4
), pp.
327
338
. 10.1080/08982110701580930
54.
Le
,
B.
,
Yvonnet
,
J.
, and
He
,
Q. C.
,
2015
, “
Computational Homogenization of Nonlinear Elastic Materials Using Neural Networks
,”
Int. J. Numer Meth. Eng.
,
104
(
12
), pp.
1061
1084
. 10.1002/nme.4953
55.
Gelman
,
A.
,
Carlin
,
J. B.
,
Stern
,
H. S.
,
Dunson
,
D. B.
,
Vehtari
,
A.
, and
Rubin
,
D. B.
,
2013
,
Bayesian Data Analysis
,
CRC Press
,
Boca Raton, FL
.
56.
Kullback
,
S.
,
1997
, Information Theory and Statistics,
Dover Publications
,
New York
.
57.
Long
,
Q.
,
Scavino
,
M.
,
Tempone
,
R.
, and
Wang
,
S.
,
2013
, “
Fast Estimation of Expected Information Gains for Bayesian Experimental Designs Based on Laplace Approximations
,”
Comput. Method Appl. M.
,
259
(
Supplement C
), pp.
24
39
. 10.1016/j.cma.2013.02.017
58.
Tierney
,
L.
, and
Kadane
,
J. B.
,
1986
, “
Accurate Approximations for Posterior Moments and Marginal Densities
,”
J. Am. Stat. Assoc.
,
81
(
393
), pp.
82
86
. 10.1080/01621459.1986.10478240
59.
Xia
,
L.
, and
Breitkopf
,
P.
,
2015
, “
Design of Materials Using Topology Optimization and Energy-Based Homogenization Approach in Matlab
,”
Struct. Multidiscip. Optim.
,
52
(
6
), pp.
1229
1241
. 10.1007/s00158-015-1294-0
60.
Andreassen
,
E.
, and
Andreasen
,
C. S.
,
2014
, “
How to Determine Composite Material Properties Using Numerical Homogenization
,”
Comp. Mater. Sci.
,
83
, pp.
488
495
. 10.1016/j.commatsci.2013.09.006
61.
Reuter
,
M.
,
Wolter
,
F.-E.
, and
Peinecke
,
N.
,
2006
, “
Laplace–Beltrami Spectra as ‘Shape-DNA’ of Surfaces and Solids
,”
Comput.-Aided Des.
,
38
(
4
), pp.
342
366
. 10.1016/j.cad.2005.10.011
62.
Lian
,
Z.
,
Godil
,
A.
,
Bustos
,
B.
,
Daoudi
,
M.
,
Hermans
,
J.
,
Kawamura
,
S.
,
Kurita
,
Y.
,
Lavoué
,
G.
,
Van Nguyen
,
H.
,
Ohbuchi
,
R.
,
Ohkita
,
Y.
,
Ohishi
,
Y.
,
Porikli
,
F.
,
Reuter
,
M.
,
Sipiran
,
I.
,
Smeets
,
D.
,
Suetens
,
P.
,
Tabia
,
H.
, and
Vandermeulen
,
D.
,
2013
, “
A Comparison of Methods for Non-Rigid 3D Shape Retrieval
,”
Pattern Recognit.
,
46
(
1
), pp.
449
461
. 10.1016/j.patcog.2012.07.014
63.
Su
,
S.
,
2010
, “
Numerical Approaches on Shape Optimization of Elliptic Eigenvalue Problems and Shape Study of Human Brains
,” Ph.D. thesis,
The Ohio State University
,
Columbus, OH
.
64.
Zhu
,
S.
,
2018
, “
Effective Shape Optimization of Laplace Eigenvalue Problems Using Domain Expressions of Eulerian Derivatives
,”
J. Optim. Theory Appl.
,
176
(
1
), pp.
17
34
. 10.1007/s10957-017-1198-9
You do not currently have access to this content.