Abstract

We present a framework for establishing credibility of a machine learning (ML) model used to predict a key process control variable setting to maximize product quality in a component manufacturing application. Our model coupled a purely data-based ML model with a physics-based adjustment that encoded subject matter expertise of the physical process. Establishing credibility of the resulting model provided the basis for eliminating a costly intermediate testing process that was previously used to determine the control variable setting.

References

1.
Kusiak
,
A.
,
2017
, “
Smart Manufacturing Must Embrace Big Data
,”
Nature
,
544
(
7648
), pp.
23
25
.10.1038/544023a
2.
U.S. FDA,
2021
, “
Artificial Intelligence and Machine Learning (AI/ML) Software as a Medical Device Action Plan
,” U.S. Food Drug Administration, Silver Spring, MD, accessed May 4, https://www.fda.gov/medical-devices/
3.
Sadagopan
,
A.
,
Huang
,
D.
,
Duzel
,
U.
,
Martin
,
L. E.
, and
Hanquist
,
K. M.
,
2021
, “
Assessment of High-Temperature Effects on Hypersonic Aerothermoelastic Analysis sing Multi-Fidelity Multi-Variate Surrogates
,”
AIAA
Paper No. 2021–1610.10.2514/6.2021-1610
4.
Westkamper
,
E.
, and
Schmidt
,
T.
,
1998
, “
Computer-Assisted Manufacturing Process Optimization With Neural Networks
,”
J. Intell. Manuf.
,
9
(
4
), pp.
289
294
.10.1023/A:1008966407212
5.
Lieber
,
D.
,
Stolpe
,
M.
,
Konrad
,
B.
,
Deuse
,
J.
, and
Morik
,
K.
,
2013
, “
Quality Prediction in Interlinked Manufacturing Processes Based on Supervised & Unsupervised Machine Learning
,”
Procedia CRIP
,
7
, pp.
193
198
.10.1016/j.procir.2013.05.033
6.
Eger
,
F.
,
Coupek
,
D.
,
Caputo
,
D.
,
Colledani
,
M.
,
Penalva
,
M.
,
Ortiz
,
J. A.
,
Freiberger
,
H.
, and
Kollegger
,
G.
,
2018
, “
Zero Defect Manufacturing Strategies for Reduction of Scrap and Inspection Effort in Multi-Stage Production Systems
,”
Procedia CRIP
,
67
, pp.
368
376
.10.1016/j.procir.2017.12.228
7.
American Society of Mechanical Engineers,
2019
, “
Standard for Verification and Validation of Computational Structural Mechanics
,” ASME, New York, Standard No. V&V10-2019.
8.
American Nuclear Society,
2016
, “
Verification and Validation of Non-Safety-Related Scientific and Engineering Computer Programs for the Nuclear Industry
,” ANS, LaGrange Park, IL, Standard No. ANSI/ANS 10.4-2008 (R2016).
9.
IEEE,
2012
, “
Standard for System and Software Verification and Validation
,” IEEE, New York, Standard No. IEEE 1012–2012.
10.
U.S. Nuclear Regulatory Commission,
2019
, “
Credibility Assessment Framework for Critical Boiling Transition Models: A Generic Safety Case to Determine the Credibility of Critical Heat Flux and Critical Power Models
,” U.S. NRC, Washington, DC, Report No.
NUREG/KM-0013
.https://www.nrc.gov/readingrm/doc-collections/nuregs/knowledge/km0013/index.html
11.
Ahn
,
J.
,
de Weck
,
O. L.
, and
Steele
,
M.
,
2014
, “
Credibility Assessment of Models and Simulations Based on NASA's Models and Simulation Standard Using the Delphi Method
,”
Syst. Eng.
,
17
(
2
), pp.
237
248
.
12.
GIA,
2021
, “
Grading the 4Cs
,” GIA, Carlsbad, CA, accessed May 11, https://4cs.gia.edu/en-us/grading-diamond-4cs/
13.
Evans
,
J. R.
,
Michael
,
S. W.
,
Meissner
,
C. A.
, and
Brandon
,
S. E.
,
2013
, “
Validating a New Assessment Method for Deception Detection: Introducing a Psychologically Based Credibility Assessment Tool
,”
J. Appl. Res. Mem. Cognit.
,
2
(
1
), pp.
33
41
.10.1016/j.jarmac.2013.02.002
14.
American Society of Mechanical Engineers,
2019
, “
Test Uncertainty
,” ASME, New York, Standard No. PTC 19.1-2018.
15.
Mandry
,
J. G.
, ,
Elele
,
J. N.
,
Hall
,
D. H.
, and
Turner
,
D. J.
,
2019
, “
Risk Assessment for Model and Simulation Credibility Characteristics
,”
ASME
Paper No. VVS2019-5152.10.1115/VVS2019-5152
16.
Bickford
,
R. L.
, and
Palnitkar
,
R. M.
,
2014
, “
Dynamic Data Filtering System and Method
,” U.S. Patent No. US8712929B1.
17.
Pedregosa
,
F.
,
2011
, “
Scikit-Learn: Machine Learning in Python
,”
J. Mach. Learn. Res.
,
12
, pp.
2825
2830
.10.5555/1953048.2078195
18.
Reed
,
R. D.
, and
Marks
,
R. J.
,
1999
,
Neural Smithing: Supervised Learning in Feedforward Artificial Neural Networks
,
Massachusetts Institute of Technology Press
,
Cambridge, MA
.
19.
Virtanen
,
P.
,
Gommers
,
R.
,
Oliphant
,
T. E.
,
Haberland
,
M.
,
Reddy
,
T.
,
Cournapeau
,
D.
,
Burovski
,
E.
,
Peterson
,
P.
,
Weckesser
,
W.
,
Bright
,
J.
,
van der Walt
,
S. J.
,
Brett
,
M.
,
Wilson
,
J.
,
Millman
,
K. J.
,
Mayorov
,
N.
,
Nelson
,
A. R. J.
,
Jones
,
E.
,
Kern
,
R.
,
Larson
,
E.
,
Carey
,
C. J.
,
Polat
,
İ.
,
Feng
,
Y.
,
Moore
,
E. W.
,
VanderPlas
,
J.
,
Laxalde
,
D.
,
Perktold
,
J.
,
Cimrman
,
R.
,
Henriksen
,
I.
,
Quintero
,
E. A.
,
Harris
,
C. R.
,
Archibald
,
A. M.
,
Ribeiro
,
A. H.
,
Pedregosa
,
F.
,
van Mulbregt
,
P.
, and
SciPy 1.0 Contributors,
2020
, “
SciPy 1.0: Fundamental Algorithms for Scientific Computing in Python
,”
Nat. Methods
,
17
(
3
), pp.
261
272
.10.1038/s41592-019-0686-2
20.
Hurley
,
N.
, and
Rickard
,
S.
,
2008
, “
Comparing Measures of Sparsity
,”
IEEE Trans. Inf. Theory
,
55
(
10
), pp.
4723
4741
.10.1109/TIT.2009.2027527
21.
Gini
,
C.
,
1912
,
Variabilità e Mutabilità
,
Libreria Eredi Virgilio Veschi
, Rome, Italy.
22.
Zhang
,
D.
, and
Lu
,
G.
,
2004
, “
Review of Shape Representation and Description Techniques
,”
Pattern Recognit.
,
37
(
1
), pp.
1
19
.10.1016/j.patcog.2003.07.008
23.
Dubuisson
,
M. P.
, and
Jain
,
A. K.
,
1994
, “
A Modified Hausdorff Distance for Object Matching
,”
Proceedings of 12th International Conference on Pattern Recognition
, Jerusalem, Israel, Oct. 9–13, pp.
566
568
.10.1109/ICPR.1994.576361
24.
Zupan
,
R.
,
2019
, “
Computational Design and Evaluation of a Smart Material Morphing Building Surface Tile
,” Ph.D. dissertation,
University of Pittsburgh
,
Pittsburgh, PA
.
You do not currently have access to this content.