Abstract

Available Gaussian processes (GPs) are inappropriate when used to globally emulate and optimize nondifferentiable functions with jump discontinuities. This is a substantial limitation as many contemporary engineering challenges are underpinned by functions with these characteristics and would benefit from a data-efficient emulator (e.g., contact problems in mechanics and material properties because of phase changes). Available GP models are inappropriate as jump discontinuities and nondifferentiabilities are local phenomena that cannot be modeled through covariance structures that describe a function’s global behavior. To overcome this limitation, we introduce the discontinuous Gaussian process (DCGP) model that involves learning a parametric, yet flexible function on the inputs to ensure that the residuals of the random process are continuous and differentiable. Through validation on a set of six test problems and one engineering problem, we show that the DCGP model outperforms available models that use multiple local emulators. The performance of the DCGP model is measured in terms of predictive fidelity and optimization efficiency. In addition, we show that there exist physical systems for which the correlation between its outputs can be modeled across discontinuities and nondifferentiabilities (i.e., globally). We believe this to be a profound insight as it suggests that statistical models, such as DCGP, could be used to expedite the discovery of emerging properties even for discontinuous and nondifferentiable functions.

References

1.
Tao
,
S.
,
Shintani
,
K.
,
Bostanabad
,
R.
,
Chan
,
Y. C.
,
Yang
,
G.
,
Meingast
,
H.
, and
Chen
,
W.
,
2017
, “
Enhanced Gaussian Process Metamodeling and Collaborative Optimization for Vehicle Suspension Design Optimization
,”
43rd Design Automation Conference, Proceedings of the ASME Design Engineering Technical Conference
,
Cleveland, OH
,
Aug. 6–9
, pp.
1
12
.
2.
Ghumman
,
U. F.
,
van Beek
,
A.
,
Munshi
,
J.
,
Chien
,
T. Y.
,
Balasubramanian
,
G.
, and
Chen
,
W.
,
2022
, “
Designing Active Layer of Organic Solar Cells Using Multi-Fidelity Molecular Simulations and Spectral Density Function
,”
Comput. Mater. Sci.
,
211
(
8
), p.
111491
.
3.
Bellamy
,
H.
,
Rehim
,
A. A.
,
Orhobor
,
O. I.
, and
King
,
R.
,
2022
, “
Batched Bayesian Optimization for Drug Design in Noisy Environments
,”
J. Chem. Inf. Model.
,
62
(
17
), pp.
3970
3981
.
4.
Williams
,
C. K. I.
, and
Rasmussen
,
C. E.
,
2006
,
Gaussian Processes for Machine Learning
, Vol. 2,
MIT Press
,
Cambridge, MA
.
5.
Giuntoli
,
A.
,
Hansoge
,
N. K.
,
van Beek
,
A.
,
Meng
,
Z.
,
Chen
,
W.
, and
Keten
,
S.
,
2021
, “
Systematic Coarse-Graining of Epoxy Resins With Machine Learning-Informed Energy Renormalization
,”
npj Comput. Math.
,
168
(
7
), pp.
1
12
.
6.
van Beek
,
A.
,
Tao
,
S.
,
Plumlee
,
M.
,
Apley
,
D. W.
, and
Chen
,
W.
,
2020
, “
Integration of Normative Decision-Making and Batch Sampling for Global Metamodeling
,”
ASME J. Mech. Des.
,
142
(
3
), p.
031114
.
7.
Bostanabad
,
R.
,
Kearney
,
T.
,
Tao
,
S.
,
Apley
,
D. W.
, and
Chen
,
W.
,
2018
, “
Leveraging the Nugget Parameter for Efficient Gaussian Process Modeling
,”
Int. J. Numer. Methods Eng.
,
114
(
5
), pp.
501
516
.
8.
Engelund
,
W. C.
,
Stanley
,
D. O.
,
Lepsch
,
R. A.
,
McMillin
,
M. M.
, and
Unal
,
R.
,
1993
, “
Aerodynamic Configuration Design Using Response Surface Methodology Analysis
,”
NASA STI/Recon Technical Report A
, Vol.
94
, p.
10718
.
9.
Friedman
,
J. H.
,
1991
, “
Multivariate Adaptive Regression Splines
,”
Ann. Stat.
,
19
(
1
), pp.
1
67
. DOI: 10.1214/aos/1176347963
10.
Hardy
,
R. L.
,
1971
, “
Multiquadric Equations of Topography and Other Irregular Surfaces
,”
J. Geophys. Res.
,
76
(
8
), pp.
1905
1915
.
11.
Wang
,
S.-C.
,
2003
,
Interdiscipl. Comput. Java Program.
, Vol.
743
,
Springer
,
Boston, MA
, pp.
81
100
.
12.
Lewis
,
K. E.
,
Chen
,
W.
, and
Schmidt
,
L. C.
,
2006
,
Decision Making in Engineering Design
,
ASME Press
,
New York
.
13.
Yao
,
J.
,
Pan
,
W.
,
Ghosh
,
S.
, and
Doshi-Velez
,
F.
,
2019
, “
Quality of Uncertainty Quantification for Bayesian Neural Network Inference
,”
arxiv
. https://arxiv.org/abs/1906.09686
14.
Choi
,
S.-K.
,
Grandhi
,
R. V.
,
Canfield
,
R. A.
, and
Pettit
,
C. L.
,
2004
, “
Polynomial Chaos Expansion With Latin Hypercube Sampling for Estimating Response Variability
,”
AIAA J.
,
42
(
6
), pp.
1191
1198
.
15.
O’Hagan
,
A.
,
2013
, “
Polynomial Chaos: A Tutorial and Critique From a Statistician’s Perspective
,”
SIAM/ASA J. Uncertainty Quantif.
,
20
(
1
), pp.
5
20
. https://www.tonyohagan.co.uk/academic/pdf/Polynomial-chaos.pdf
16.
Branson
,
Z.
,
Rischard
,
M.
,
Bornn
,
L.
, and
Miratrix
,
L.
,
2019
, “
A Nonparametric Bayesian Methodology for Regression Discontinuity Designs
,”
J. Stat. Plann. Inference
,
202
(
1
), pp.
14
30
.
17.
Wu
,
X.
,
2022
, “
Hierarchical Gaussian Process Models for Regression Discontinuity/Kink under Sharp and Fuzzy Designs
,”
arxiv
. https://arxiv.org/abs/2110.00921
18.
Ornstein
,
J.
, and
Duck-Mary
,
B.
,
2023
, Gaussian Process Regression Discontinuity, https://joeornstein.github.io/publications/gprd.pdf
19.
Moustapha
,
M.
, and
Sudret
,
B.
,
2023
, “
Learning Non-stationary and Discontinuous Functions Using Clustering, Classification and Gaussian Process Modelling
,”
Comput. Struct.
,
281
(
6
), p.
107035
.
20.
Biswas
,
A.
, and
Hoyle
,
C.
,
2021
, “
An Approach to Bayesian Optimization for Design Feasibility Check on Discontinuous Black-Box Functions
,”
ASME J. Mech. Des.
,
143
(
3
), p.
031716
.
21.
Jetton
,
C.
,
Campbell
,
M.
, and
Hoyle
,
C.
,
2024
, “
Constraining the Feasible Design Space in Bayesian Optimization With User Feedback
,”
ASME J. Mech. Des.
,
146
(
4
), p.
041703
.
22.
Ran
,
Z.
,
Zhang
,
Z.
,
Tian
,
Y.
, and
Li
,
J.
,
2023
, “
Accelerated High-Entropy Alloys Discovery for Electrocatalysis Via Robotic-Aided Active Learning
,”
NeurIPS 2023 Workshop on Adaptive Experimental Design and Active Learning in the Real World
,
New Orleans, LA
,
Dec. 10–16
, pp.
1
9
.
23.
van Beek
,
A.
,
Ghumman
,
U. F.
,
Munshi
,
J.
,
Tao
,
S.
,
Chien
,
T. Y.
,
Balasubramanian
,
G.
,
Plumlee
,
M.
,
Apley
,
D.
, and
Chen
,
W.
,
2021
, “
Scalable Adaptive Batch Sampling in Simulation-Based Design With Heteroscedastic Noise
,”
ASME J. Mech. Des.
,
143
(
3
), p.
031709
.
24.
Ath
,
G. D.
,
Everson
,
R. M.
, and
Fieldsend
,
J. E.
,
2021
, “
How Bayesian Should Bayesian Optimisation Be?
Proceedings of the Genetic and Evolutionary Computation Conference Companion
,
Lille, France
,
July 10–14
, pp.
1860
1869
.
25.
Stromberg
,
K. R.
,
2015
,
An Introduction to Classical Real Analysis
,
American Mathematical Soc
,
Providence, RI
.
26.
Murphy
,
I. S.
,
1982
, “
Karl R. Stromberg, An Introduction to Classical Real Analysis
,”
Proc. Edinburgh Math. Soc.
,
25
(
1
), pp.
105
106
.
27.
Plumlee
,
M.
, and
Apley
,
D.
,
2016
, “
Lifted Brownian Kriging Models
,”
Technometrics
,
59
(
2
), pp.
1
10
. doi.org/10.1080/00401706.2016.1211555
28.
van Beek
,
A.
,
2025
, “
Bayesian Optimization for Intrinsically Noisy Response Surfaces
,”
arxiv
. https://arxiv.org/abs/2503.00327
29.
Seepersad
,
C. C.
,
Pedersen
,
K.
,
Emblemsvåg
,
J.
,
Bailey
,
R.
,
Allen
,
J. K.
, and
Mistree
,
F.
,
2006
, “Chapter 25: The Validation Square: How Does One Verify and Validate a Design Method?,”
Decision Making in Engineering Design
, pp.
303
314
.
30.
Forrester
,
A. I. J.
, and
Keane
,
A. J.
,
2009
, “
Recent Advances in Surrogate-Based Optimization
,”
Prog. Aerosp. Sci.
,
45
(
1–3
), pp.
50
79
.
31.
Nair
,
V.
, and
Hinton
,
G. E.
,
2010
, “
Rectified Linear Units Improve Restricted Boltzmann Machines
,”
Proceedings of the 27th International Conference on Machine Learning
,
Haifa, Israel
,
June 21–24
, pp.
807
814
.
32.
Wuttig
,
M.
, and
Yamada
,
N.
,
2007
, “
Phase-Change Materials for Rewriteable Data Storage
,”
Nat. Mater.
,
6
(
11
), pp.
824
832
.
33.
Forrester
,
A.
,
Sobester
,
A.
, and
Keane
,
A.
,
2008
,
Engineering Design Via Surrogate Modelling: A Practical Guide
,
John Wiley & Sons
,
Chippenham
.
34.
Molga
,
M.
, and
Smutnicki
,
C.
,
2005
, Test Functions for Optimization Needs, https://marksmannet.com/RobertMarks/Classes/ENGR5358/Papers/functions.pdf
35.
Zhang
,
S.
,
Zhu
,
P.
,
Chen
,
W.
, and
Arendt
,
P.
,
2013
, “
Concurrent Treatment of Parametric Uncertainty and Metamodeling Uncertainty in Robust Design
,”
Struct. Multidiscipl. Optim.
,
47
(
1
), pp.
63
76
.
36.
Colás
,
R.
, and
Totten
,
G. E.
,
2016
,
Encyclopedia of Iron, Steel, and Their Alloys
,
CRC Press
,
Boca Raton, FL
.
You do not currently have access to this content.