Validation assesses the accuracy of a mathematical model by comparing simulation results to experimentally measured quantities of interest. Model validation experiments emphasize obtaining detailed information on all input data needed by the mathematical model, in addition to measuring the system response quantities (SRQs) so that the predictive accuracy of the model can be critically determined. This article proposes a framework for assessing model validation experiments for computational fluid dynamics (CFD) regarding information content, data completeness, and uncertainty quantification (UQ). This framework combines two previously published concepts: the strong-sense model validation experiments and the modeling maturity assessment procedure referred to as the predictive capability maturity method (PCMM). The model validation experiment assessment requirements are captured in a table of six attributes: experimental facility, analog instrumentation and signal processing, boundary and initial conditions, fluid and material properties, test conditions, and measurement of system responses, with four levels of information completeness for each attribute. The specifics of this table are constructed for a generic wind tunnel experiment. Each attribute’s completeness is measured from the perspective of the level of detail needed for input data using direct numerical simulation of the Navier–Stokes equations. While this is an extraordinary and unprecedented requirement for level of detail in a model validation experiment, it is appropriate for critical assessment of modern CFD simulations.

References

References
1.
Oberkampf
,
W. L.
, and
Aeschliman
,
D. P.
,
1992
, “
Joint Computational/Experimental Aerodynamics Research on a Hypersonic Vehicle—Part 1: Experimental Results
,”
AIAA J.
,
30
(
8
), pp.
2000
2009
.
2.
Marvin
,
J. G.
,
1995
, “
Perspective on Computational Fluid Dynamics Validation
,”
AIAA J.
,
33
(
10
), pp.
1778
1787
.
3.
Aeschliman
,
D. P.
, and
Oberkampf
,
W. L.
,
1998
, “
Experimental Methodology for Computational Fluid Dynamics Code Validation
,”
AIAA J.
,
36
(
5
), pp.
733
741
.
4.
Oberkampf
,
W. L.
,
Trucano
,
T. G.
, and
Hirsch
,
C.
,
2004
, “
Verification, Validation, and Predictive Capability in Computational Engineering and Physics
,”
ASME Appl. Mech. Rev.
,
57
(
5
), pp.
345
384
.
5.
Oberkampf
,
W. L.
,
Pilch
,
M.
, and
Trucano
,
T. G.
,
2007
, “
Predictive Capability Maturity Model for Computational Modeling and Simulation
,” Sandia National Laboratories, Albuquerque, NM, Report No.
SAND2007-5948
.
6.
Oberkampf
,
W. L.
, and
Roy
,
C. J.
,
2010
,
Verification and Validation in Scientific Computing
,
Cambridge University Press
,
Cambridge, UK
.
7.
Oberkampf
,
W. L.
, and
Trucano
,
T. G.
,
2002
, “
Verification and Validation in Computational Fluid Dynamics
,”
Prog. Aerosp. Sci.
,
38
(
3
), pp.
209
272
.
8.
Trucano
,
T. G.
,
Pilch
,
M.
, and
Oberkampf
,
W. L.
,
2002
, “
General Concepts for Experimental Validation of ASCI Code Applications
,” Sandia National Laboratories, Albuquerque, NM, Report No.
SAND2002-0341
.
9.
NEA
,
1977
, “
Loss of Coolant Accident Standard Problems
,” Nuclear Energy Agency, Committee on the Safety of Nuclear Installations, Paris, France.
10.
CSNI
,
2004
, “
CSNI International Standard Problem Procedures
,” Nuclear Energy Agency, Committee on the Safety of Nuclear Installations, Paris, France, Revision 4, CSNI Report No.
17
.
11.
NPARC
,
2000
, “
CFD Verification and Validation: NPARC Alliance
,” NASA Glenn Research Center, Cleveland, OH, accessed Jan. 15, 2017, http://www.grc.nasa.gov/WWW/wind/valid/homepage.html
12.
ERCOFTAC
,
2017
, “
European Research Community on Flow, Turbulence and Combustion
,” European Research Community on Flow, Turbulence and Combustion, London, accessed Jan. 15, 2017, http://www.ercoftac.org/
13.
QNET-CFD
,
2003
, “
The ERCOFTAC QNET-CFD Wiki Home Page
,” European Union Network on Quality and Trust in the Industrial Application of Computational Fluid Dynamics, accessed Sept. 1, 2003, http://qnet-ercoftac.cfms.org.uk/w/index.php/Main_Page
14.
Rizzi
,
A.
, and
Vos
,
J.
,
1998
, “
Toward Establishing Credibility in Computational Fluid Dynamics Simulations
,”
AIAA J.
,
36
(
5
), pp.
668
675
.
15.
Vos
,
J. B.
,
Rizzi
,
A.
,
Darracq
,
D.
, and
Hirschel
,
E. H.
,
2002
, “
Navier-Stokes Solvers in European Aircraft Design
,”
Prog. Aerosp. Sci.
,
38
(
8
), pp.
601
697
.
16.
Oberkampf
,
W. L.
,
Aeschliman
,
D. P.
,
Tate
,
R. E.
, and
Henfling
,
J. F.
,
1993
, “
Experimental Aerodynamics Research on a Hypersonic Vehicle
,” Sandia National Laboratories, Albuquerque, NM, Report No.
SAND-92-1411
.
17.
Cosner
,
R. R.
,
1995
, “
CFD Validation Requirements for Technology Transition
,”
AIAA
Paper No. 95-2227.
18.
Oberkampf
,
W. L.
,
Aeschliman
,
D. P.
,
Henfling
,
J. F.
, and
Larson
,
D. E.
,
1995
, “
Surface Pressure Measurements for CFD Code Validation in Hypersonic Flow
,”
AIAA
Paper No. 95-2273.
19.
Porter
,
J. L.
,
1996
, “
A Summary Overview of Selected Computational Fluid Dynamics (CFD) Code Validation/Calibration Activities
,”
AIAA
Paper No. 96-2053.
20.
Barber
,
T. J.
,
1998
, “
Role of Code Validation and Certification in the Design Environment
,”
AIAA J.
,
36
(
5
), pp.
752
758
.
21.
Benek
,
J. A.
,
Kraft
,
E. M.
, and
Lauer
,
R. F.
,
1998
, “
Validation Issues for Engine—Airframe Integration
,”
AIAA J.
,
36
(
5
), pp.
759
764
.
22.
Roache
,
P. J.
,
1998
,
Verification and Validation in Computational Science and Engineering
,
Hermosa Publishers
,
Albuquerque, NM
.
23.
NASA
,
2008
, “
Standard for Models and Simulations
,” National Aeronautics and Space Administration, Washington, DC, Standard No.
NASA-STD-7009A
.
24.
Kaizer
,
J. S.
,
Heller
,
A. K.
, and
Oberkampf
,
W. L.
,
2015
, “
Scientific Computer Simulation Review
,”
Reliab. Eng. Syst. Saf.
,
138
, pp.
210
218
.
25.
Ferson
,
S.
, and
Ginzburg
,
L. R.
,
1996
, “
Different Methods are Needed to Propagate Ignorance and Variability
,”
Reliab. Eng. Syst. Saf.
,
54
(
2–3
), pp.
133
144
.
26.
Ferson
,
S.
,
Kreinovich
,
V.
,
Ginzburg
,
L.
,
Myers
,
D. S.
, and
Sentz
,
K.
,
2002
, “
Constructing Probability Boxes and Dempster-Shafer Structures
,” Sandia National Laboratories, Albuquerque, NM, Report No.
SAND2002-4015
.
27.
ANSI
,
1997
, “
U.S. Guide to the Expression of Uncertainty in Measurement
,” American National Standards Institute, Boulder, CO.
28.
AIAA
,
1999
, “
Assessment of Experimental Uncertainty With Application to Wind Tunnel Testing
,” American Institute of Aeronautics and Astronautics, Reston, VA, Standard No.
AIAA S-071A-1999
.
29.
Beckwith
,
T. G.
,
Marangoni
,
R. D.
, and
Lienhard
,
J. H.
,
2006
,
Mechanical Measurements
,
Prentice Hall
, Upper Saddle River, NJ.
30.
Coleman
,
H. W.
, and
Steele
,
W. G.
, Jr.
,
2009
,
Experimentation, Validation, and Uncertainty Analysis for Engineers
,
Wiley
,
New York
.
31.
Wu
,
C. F. J.
, and
Hamada
,
M. S.
,
2009
,
Experiments: Planning, Analysis, and Optimization
,
Wiley
,
Hoboken, NJ
.
32.
Rhode
,
M. N.
, and
Oberkampf
,
W. L.
,
2017
, “
Estimation of Uncertainties for a Model Validation Experiment in a Wind Tunnel
,”
J. Spacecr. Rockets
,
54
(
1
), pp.
155
168
.
33.
Montgomery
,
D. C.
,
2001
,
Design and Analysis of Experiments
,
Wiley
,
Hoboken, NJ
.
34.
Box
,
G. E. P.
,
Hunter
,
J. S.
, and
Hunter
,
W. G.
,
2005
,
Statistics for Experimenters: Design, Innovation, and Discovery
,
Wiley
,
New York
.
35.
Hinkelmann
,
K.
, and
Kempthorne
,
O.
,
2008
,
Design and Analysis of Experiments
(Introduction to Experimental Design), Vol.
1
,
Wiley
,
Hoboken, NJ
.
36.
Salicone
,
S.
,
2007
,
Measurement Uncertainty: An Approach Via the Mathematical Theory of Evidence
,
Springer
,
Berlin
.
37.
Drosg
,
M.
,
2009
,
Dealing With Uncertainties: A Guide to Error Analysis
,
Springer
,
Berlin
.
38.
Grabe
,
M.
,
2014
,
Measurement Uncertainties in Science and Technology
,
Springer
,
Berlin
.
You do not currently have access to this content.