Model verification and validation (V&V) remain a critical step in the simulation model development process. A model requires verification to ensure that it has been correctly transitioned from a conceptual form to a computerized form. A model also requires validation to substantiate the accurate representation of the system it is meant to simulate. Validation assessments are complex when the system and model both generate high-dimensional functional output. To handle this complexity, this paper reviews several wavelet-based approaches for assessing models of this type and introduces a new concept for highlighting the areas of contrast and congruity between system and model data. This concept identifies individual wavelet coefficients that correspond to the areas of discrepancy between the system and model.

References

References
1.
Balci
,
O.
,
1997
, “
Verification Validation and Accreditation of Simulation Models
,”
Winter Simulation Conference
, pp.
135
141
.
2.
Balci
,
O.
,
2013
,
Encyclopedia of Operations Research and Management Science
,
3rd ed.
,
Springer
,
New York
, pp.
1618
1627
.
3.
Atkinson
,
A.
,
Hill
,
R.
,
Pignatiello
,
J.
,
Vining
,
G.
,
White
,
E.
, and
Chicken
,
E.
,
2017
, “
Dynamic Model Validation Metric Based on Wavelet Thresholded Signals
,”
J. Verif., Validation Uncertainty Quantif.
,
2
(
2
), p.
021002
.
4.
Atkinson
,
A.
,
Hill
,
R.
,
Pignatiello
,
J.
,
Vining
,
G.
,
White
,
E.
, and
Chicken
,
E.
,
2017
, “
Wavelet Anova Approach to Model Validation
,”
Simul. Modell. Pract. Theory
,
78
, pp.
18
27
.
5.
Atkinson
,
A.
,
Hill
,
R.
,
Pignatiello
,
J.
,
Vining
,
G.
,
White
,
E.
, and
Chicken
,
E.
,
2018
, “
Wavelet Anova Bisection Method for Identifying Simulation Model Bias
,”
Simul. Modell. Pract. Theory
,
80
, pp.
66
74
.
6.
Law
,
A. M.
,
2013
,
Simulation Modeling and Analysis
,
5th ed.
,
McGraw-Hill
,
New York
.
7.
Schwer
,
L.
,
Mair
,
H.
, and
Crane
,
R.
,
2006
,
Guide for Verification and Validation in Computational Solid Mechanics
,
The American Society of Mechanical Engineers
,
New York
.
8.
McHale
,
M.
,
Friedman
,
J.
, and
Karian
,
J.
,
2009
, “
Standard for Verification and Validation in Computational Fluid Dynamics and Heat Transfer
,” The American Society of Mechanical Engineers, New York Standard No.
VV20-2009
.https://www.asme.org/products/codes-standards/v-v-20-2009-standard-verification-validation
9.
Fishman
,
G. S.
, and
Kiviat
,
P. J.
,
1967
, “
The Analysis of Simulation-Generated Time Series
,”
Manage. Sci.
,
13
(
7
), pp.
525
557
.
10.
Kleijnen
,
J. P.
,
1995
, “
Verification and Validation of Simulation Models
,”
Eur. J. Oper. Res.
,
82
(
1
), pp.
145
162
.
11.
Robinson
,
S.
,
2004
,
Simulation: The Practice of Model Development and Use
,
1st ed.
,
Wiley
,
Chichester, UK
.
12.
Sargent
,
R. G.
,
2013
, “
Verification and Validation of Simulation Models
,”
J. Simul.
,
7
(
1
), pp.
12
24
.
13.
Shannon
,
R. E.
,
1975
,
Systems Simulation: The Art and Science
,
1st ed.
,
Prentice Hall
,
Englewood Cliffs, NJ
.
14.
Box
,
G. E.
,
Jenkins
,
G. M.
, and
Reinsel
,
G. C.
,
2008
,
Time Series Analysis: Forecasting and Control
,
4th ed.
,
Wiley
,
New York
.
15.
Geers
,
T. L.
,
1984
, “
An Objective Error Measure for the Comparison of Calculated and Measured Transient Response Histories
,”
Shock Vib. Bull.
,
54
(2), pp.
99
108
.https://archive.org/details/DTIC_ADA148132
16.
Russell
,
D. M.
,
1997
, “
Error Measures for Comparing Transient Data—Part I: Development of a Comprehensive Error Measure
,”
68th Shock and Vibration Symposium
, Hunt Valley, MD, Nov. 3–7, pp.
175
184
.
17.
Sargent
,
R. G.
,
2003
, “
Verification and Validation of Simulation Models
,”
Winter Simulation Conference
, pp.
37
48
.
18.
Ramsay
,
J. O.
, and
Silverman
,
B. W.
,
2005
,
Functional Data Analysis
,
2nd ed.
,
Springer Science Business Media
,
New York
.
19.
McKay
,
J. L.
,
Welch
,
T. D.
,
Vidakovic
,
B.
, and
Ting
,
L. H.
,
2013
, “
Statistically Significant Contrasts Between EMG Waveforms Revealed Using Wavelet-Based Functional Anova
,”
J. Neurophysiol.
,
109
(
2
), pp.
591
602
.
20.
Park
,
B.
, and
Schneeberger
,
J.
,
2003
, “
Microscopic Simulation Model Calibration and Validation: Case Study of VISSIM Simulation Model for a Coordinated Actuated Signal System
,”
Transp. Res. Rec.: J. Transp. Res. Board
,
1856
, pp.
185
192
.
21.
Godley
,
S. T.
,
Triggs
,
T. J.
, and
Fildes
,
B. N.
,
2002
, “
Driving Simulator Validation for Speed Research
,”
Accid. Anal. Prev.
,
34
(
5
), pp.
589
600
.
22.
Sarin
,
H.
,
Kokkolaras
,
M.
,
Hulbert
,
G.
,
Papalambros
,
P.
,
Barbat
,
S.
, and
Yang
,
R.-J.
,
2010
, “
Comparing Time Histories for Validation of Simulation Models: Error Measures and Metrics
,”
ASME J. Dyn. Syst. Meas. Control
,
132
(
6
), p.
061401
.
23.
Cheng
,
Z.
,
Pellettiere
,
J.
, and
Wright
,
N.
,
2006
, “
Wavelet-Based Test-Simulation Correlation Analysis for the Validation of Biodynamical Modeling
,”
24th Conference and Exposition on Structural Dynamics
, St Louis, MI, Jan 30–Feb. 2, pp.
2124
2132
.
24.
Harmon
,
S.
, and
Youngblood
,
S. M.
,
2005
, “
A Proposed Model for Simulation Validation Process Maturity
,”
J. Defense Model. Simul.: Appl., Methodol., Technol.
,
2
(
4
), pp.
179
190
.
25.
Burrus
,
C. S.
,
Gopinath
,
R. A.
, and
Guo
,
H.
,
1998
,
Introduction to Wavelets and Wavelet Transforms
,
1st ed.
,
Prentice Hall
,
Upper Saddle River, NJ
.
26.
Chui
,
C. K.
,
1992
,
An Introduction to Wavelets
,
1st ed.
,
Academic Press
,
Boston, MA
.
27.
Najmi
,
A.-H.
,
2012
,
Wavelets a Concise Guide
,
1st ed.
,
Johns Hopkins University Press
,
Baltimore, MD
.
28.
Ogden
,
T.
,
1997
,
Essential Wavelets for Statistical Applications and Data Analysis
,
1st ed.
,
Birkhauser
,
Boston, MA
.
29.
Donoho
,
D. L.
, and
Johnstone
,
J. M.
,
1994
, “
Ideal Spatial Adaptation by Wavelet Shrinkage
,”
Biometrika
,
81
(
3
), pp.
425
455
.
30.
Girimurugan
,
S.
,
Chicken
,
E.
,
Pignatiello
,
J. J.
, Jr.
, and
Zeisset
,
M. S.
,
2013
,
Industrial and Systems Engineering Research Conference
, San Juan, PR, May 18–22, pp.
3235
3244
.
31.
Girimurugan
,
S. B.
,
2014
, “
Nonlinear Multivariate Tests for High-Dimensional Data Using Wavelets With Applications in Genomics and Engineering
,” Ph.D. thesis, Florida State University, Tallahassee, FL.
32.
Bracey
,
M. J.
,
2016
, “
Dynamic Modeling of Thermal Management System With Exergy Based Optimization
,”
Master's thesis
, Wright State University, Dayton, OH.https://etd.ohiolink.edu/!etd.send_file?accession=wright1503682474459341&disposition=inline
You do not currently have access to this content.