Various stochastic validation metrics have been developed for validating models, among which area metric is frequently used in many practical problems. However, the existing area metric does not consider experimental epistemic uncertainty caused by lack of sufficient physical observations. Therefore, it cannot provide a confidence level associated with the amount of experimental data, which is a desired characteristic of validation metric. In this paper, the concept of area metric is extended to a new metric, namely interval area metric, for single-site model validation with limited experimental data. The kernel of the proposed metric is defining two boundary distribution functions based on Dvoretzky–Kiefer–Wolfowitz inequality, so as to provide an interval at a given confidence level, which covers the true cumulative distribution function (CDF) of physical observations. Based on this interval area metric, the validity of a model can be quantitatively measured with the specific confidence level in association with consideration of the lack of experiment information. The new metric is examined and compared with the existing metrics through numerical case studies to demonstrate its validity and discover its properties. Furthermore, an engineering example is provided to illustrate the effectiveness of the proposed metric in practical satellite structure engineering application.

References

References
1.
Malak
,
R. J.
, and
Paredis
,
C. J. J.
,
2010
, “
Using Support Vector Machines to Formalize the Valid Input Domain of Predictive Models in Systems Design Problems
,”
ASME J. Mech. Des.
,
132
(
10
), p.
101001
.
2.
Oberkampf
,
W. L.
,
Sindir
,
M. N.
, and
Terrence Conlisk
,
A.
,
1998
, “
Guide for the Verification and Validation of Computational Fluid Dynamics Simulations
,”
American Institute of Aeronautics and Astronautics, Reston, VA, AIAA Standard
No.
G-077-1998
.
3.
Liu
,
Y.
,
Chen
,
W.
,
Arendt
,
P.
, and
Huang
,
H. Z.
,
2011
, “
Toward a Better Understanding of Model Validation Metrics
,”
ASME J. Mech. Des.
,
133
(
7
), p.
071005
.
4.
Ferson
,
S.
,
Oberkampf
,
W. L.
, and
Ginzburg
,
L.
,
2008
, “
Model Validation and Predictive Capability for the Thermal Challenge Problem
,”
Comput. Methods Appl. Mech. Eng.
,
197
(
29–32
), pp.
2408
2430
.
5.
Choudhary
,
A.
,
Voyles
,
I. T.
,
Roy
,
C. J.
,
Oberkampf
,
W. L.
, and
Patil
,
M.
,
2016
, “
Probability Bounds Analysis Applied to the Sandia Verification and Validation Challenge Problem
,”
ASME J. Verif. Validation Uncertainty Quantif.
,
1
(
1
), p.
011003
.
6.
Li
,
W.
,
Chen
,
S.
,
Jiang
,
Z.
,
Apley
,
D. W.
,
Lu
,
Z.
, and
Chen
,
W.
,
2016
, “
Integrating Bayesian Calibration, Bias Correction, and Machine Learning for the 2014 Sandia Verification and Validation Challenge Problem
,”
ASME J. Verif. Validation Uncertainty Quantif.
,
1
(
1
), p.
011004
.
7.
Xi
,
Z.
, and
Yang
,
R.-J.
,
2015
, “
Reliability Analysis With Model Uncertainty Coupling With Parameter and Experiment Uncertainties: A Case Study of 2014 Verification and Validation Challenge Problem
,”
ASME J. Verif. Validation Uncertainty Quantif.
,
1
(
1
), p.
011005
.
8.
Lima
,
E. A. B. F.
,
Oden
,
J. T.
,
Wohlmuth
,
B.
,
Shahmoradi
,
A.
,
Hormuth
,
D. A.
,
Yankeelov
,
T. E.
,
Scarabosio
,
L.
, and
Horger
,
T.
,
2017
, “
Selection and Validation of Predictive Models of Radiation Effects on Tumor Growth Based on Noninvasive Imaging Data
,”
Comput. Methods Appl. Mech. Eng.
,
327
, pp.
277
305
.
9.
Jung
,
B. C.
,
Park
,
J.
,
Oh
,
H.
,
Kim
,
J.
, and
Youn
,
B. D.
,
2015
, “
A Framework of Model Validation and Virtual Product Qualification With Limited Experimental Data Based on Statistical Inference
,”
Struct. Multidiscip. Optim.
,
51
(
3
), pp.
573
583
.
10.
Wang
,
Z.
,
Fu
,
Y.
,
Yang
,
R. J.
,
Barbat
,
S.
, and
Chen
,
W.
,
2016
, “
Validating Dynamic Engineering Models Under Uncertainty
,”
ASME J. Mech. Des.
,
138
(
11
), p.
111402
.
11.
Ling
,
Y.
, and
Mahadevan
,
S.
,
2013
, “
Quantitative Model Validation Techniques: New Insights
,”
Reliab. Eng. Syst. Saf.
,
111
, pp.
217
231
.
12.
Voyles
,
I. T.
, and
Roy
,
C. J.
,
2014
, “
Evaluation of Model Validation Techniques in the Presence of Uncertainty
,”
AIAA
Paper No. 2014-0120.
13.
Li
,
W.
,
Chen
,
W.
,
Jiang
,
Z.
,
Lu
,
Z.
, and
Liu
,
Y.
,
2014
, “
New Validation Metrics for Models With Multiple Correlated Responses
,”
Reliab. Eng. Syst. Saf.
,
127
, pp.
1
11
.
14.
Pradlwarter
,
H. J.
, and
Schuëller
,
G. I.
,
2008
, “
The Use of Kernel Densities and Confidence Intervals to Cope With Insufficient Data in Validation Experiments
,”
Comput. Methods Appl. Mech. Eng.
,
197
(
29–32
), pp.
2550
2560
.
15.
Ao
,
D.
,
Hu
,
Z.
, and
Mahadevan
,
S.
,
2017
, “
Dynamics Model Validation Using Time-Domain Metrics
,”
ASME J. Verif. Validation Uncertainty Quantif.
,
2
(
1
), p.
011004
.
16.
Roy
,
C. J.
, and
Oberkampf
,
W. L.
,
2011
, “
A Comprehensive Framework for Verification, Validation, and Uncertainty Quantification in Scientific Computing
,”
Comput. Methods Appl. Mech. Eng.
,
200
(
25–28
), pp.
2131
2144
.
17.
Beer
,
M.
,
Ferson
,
S.
, and
Kreinovich
,
V. Y.
,
2013
, “
Imprecise Probabilities in Engineering Analyses
,”
Mech. Syst. Signal Process.
,
37
(
1–2
), pp.
4
29
.
18.
Oberkampf
,
W. L.
, and
Roy
,
C. J.
,
2010
,
Verification and Validation in Scientific Computing
, Cambridge University Press, Cambridge, UK.
19.
Ferson
,
S.
,
Kreinovick
,
V.
,
Ginzburg
,
L.
, and
Sentz
,
F.
,
2003
, “
Constructing Probability Boxes and Dempster-Shafer Structures
,” Sandia National Laboratories, Albuquerque, NM, Report No.
SAND2002-4015
.
20.
Van der Vaart
,
A. W.
,
1998
,
Asymptotic Statistics
,
Cambridge University Press
, Cambridge, UK.
21.
Xiong
,
Y.
,
Chen
,
W.
,
Tsui
,
K. L.
, and
Apley
,
D. W.
,
2009
, “
A Better Understanding of Model Updating Strategies in Validating Engineering Models
,”
Comput. Methods Appl. Mech. Eng.
,
198
(
15–16
), pp.
1327
1337
.
22.
Oliver
,
T. A.
,
Terejanu
,
G.
,
Simmons
,
C. S.
, and
Moser
,
R. D.
,
2015
, “
Validating Predictions of Unobserved Quantities
,”
Comput. Methods Appl. Mech. Eng.
,
283
, pp.
1310
1335
.
You do not currently have access to this content.