The continued advancements in high performance computing, both in hardware and software, and the cost competitive advantage of numerical simulations over laboratory experiments have made computational fluid dynamics an integral tool in the study of science and engineering problems. However, unlike experimental methods for which a large body of knowledge and techniques exist for the evaluation of experimental error and uncertainty (and are widely accepted), equivalent techniques for the evaluation of numerical error and uncertainty are less well developed and accepted. A recent article in the Journal of Fluids Engineering (Celik, 1993) highlighted this point and identified possible reasons for both the lack of interest in this topic and lack of accepted methods for the evaluation of uncertainty by the computational science community. Celik went on to identify three major topics of relevance to the issue of numerical accuracy and uncertainty. These are: (i) the separation of numerical errors from modeling errors; (ii) the identification, estimation, and reduction of numerical errors; and (iii) the assessment of codes and computational schemes with respect to numerical uncertainty through benchmarking. As Celik indicated, there is the need to improve the quality of the large number of papers being published today in computational fluid dynamics, and thus there is an urgent need for implementing a policy regarding numerical uncertainty analysis or quantitative error estimation.
The Fluids Engineering Division (FED) of the ASME, through the Coordinating Group on Computational Fluid Dynamics (CGCFD), has taken on the task of leading the computational fluid dynamics community and focusing their attention on the formulation of reasonable measures for numerical accuracy. The CGCFD has by charter the responsibility to promote discussion and interest in research into all areas of computational fluid dynamics and principal among these are methods for the evaluation of numerical accuracy. The CGCFD has performed its function by conducting symposia, forums, and panel discussions addressing this complex topic. The objective here is to delineate standard practices by which computational studies may be performed and the standards by which archival publications will be gauged. As a result of these meetings and discussions a new level of standards for the evaluation of journal publications has been promulgated for the Journal of Fluids Engineering.
The Journal of Fluids Engineering has had the policy that it: will not accept for publication any paper reporting the numerical solution of a fluids engineering problem that fails to address the task of systematic truncation error testing and accuracy estimation. This policy statement, originally presented in Roache, Ghia, and White (1986), was the first of its kind and provided a very general standard for evaluating journal publications. In the seven years since this policy's implementation, significant advances in computer hardware and computational software have occurred such that CFD is no longer in its infancy, but is a full-fledged tool in engineering and scientific problem solving. As Roache, Ghia, and White (1986) point out, fifteen years before the publication of their editorial, any successful calculation was of interest, and much of this exploratory work deserved publication. But even in their time it was recognized that this practice was outmoded. Therefore, it seems only logical and scientifically correct that the CFD community meet higher standards for evaluation of accuracy, standards on par with those required of the experimental community. What follows below then, are refinements and enhancements to the Journal's original statement on numerical accuracy, which attempt to elucidate the criteria by which Journal papers will be judged.
Finally, it is not the intent of this new policy statement to eliminate a class of simulations which some have referred to as “practical engineering project simulations.” The justification by these individuals for performing a single grid simulation has been that budget constraints, schedule constraints, or computer resource constraints prevent a systematic analysis of accuracy from being performed. It is assumed that in performing CFD analyses for “practical engineering projects,” for which experimental data is usually not available, that one must perform, in the natural course of the project, an evaluation of the accuracy of the simulation results in order to determine the validity of these particular calculations. Without such an effort there is no clear justification for presenting a simulation as representative of the physical phenomena. Therefore, it would seem only natural, even in the solution of practical engineering problems, that the items addressed here, be used to validate a simulation.
The author wishes to acknowledge the comments and suggestions to this new policy statement from Prof. B. P. Leonard (University of Akron), Dr. P. J. Roache (Ecodynamics), Dr. Ramesh Agarwal (McDonnell Douglas), Prof. Oktay Baysal (Old Dominion University), and Prof Demetri Telionis (Virginia Polytechnic Institute and State University).
The guidelines on the following page have been approved by the FED Coordinating Group on Computational Fluid Mechanics and the Editorial Board of the Journal.
Journal of Fluids Engineering Editorial PolicyStatement on the Control of Numerical Accuracy
Although no standard method for evaluating numerical uncertainty is currently accepted by the CFD community, there are numerous methods and techniques available to the user to accomplish this task. The following is a list of guidelines, enumerating the criteria to be considered for archival publication of computational results in the Journal of Fluids Engineering
- (1)
Authors must be precise in describing the numerical method used; this includes an assessment of the formal order of accuracy of the truncation error introduced by individual terms in the governing equations, such as diffusive terms, source terms, and most importantly, the convective terms. It is not enough to state, for example, that the method is based on a “conservative finite-volume formulation,” giving then a reference to a general CFD textbook.
- (2)
The numerical method used must be at least formally second-order accurate in space (based on a Taylor series expansion) for nodes in the interior of the computational grid. The computational expense of second, third, and higher order methods are more expensive (per grid point) than first order schemes, but the computational efficiency of these higher order methods (accuracy per overall cost) is much greater. And, it has been demonstrated many times that, for first order methods, the effect of numerical diffusion on the solution accuracy is devastating.
- (3)
Methods using a blending or switching strategy between first and second order methods (in particular, the well-known “hybrid,” “power-law,” and related exponential schemes) will be viewed as first-order methods, unless it can be demonstrated that their inherent numerical diffusion does not swamp or replace important modelled physical diffusion terms. A similar policy applies to methods invoking significant amounts of explicitly added artificial viscosity or diffusivity.
- (4)
Solutions over a range of significantly different grid resolutions should be presented to demonstrate grid-independent or grid-convergent results. This criterion specifically addresses the use of improved grid resolution to systematically evaluate trucation error and accuracy. The use of error estimates based on methods such as Richardson extrapolation or those techniques now used in adaptive grid methods, may also be used to demonstrate solution accuracy.
- (5)
Stopping criteria for iterative calculations need to be precisely explained. Estimates must be given for the corresponding convergence error.
- (6)
In time-dependent solutions, temporal accuracy must be demonstrated so that the spurious effects of phase error are shown to be limited. In particular, it should be demonstrated that unphysical oscillations due to numerical dispersion are significantly smaller in amplitude than captured short-wavelength (in time) features of the flow.
- (7)
Clear statements defining the methods used to implement boundary and initial conditions must be presented. Typically, the overall accuracy of a simulation is strongly affected by the implementation and order of the boundary conditions. When appropriate, particular attention should be paid to the treatment of inflow and outflow boundary conditions.
- (8)
In the presentation of an existing algorithm or code, all pertinent references or other publications must be cited in the paper, thus aiding the reader in evaluating the code and its method without the need to redefine details of the methods in the current paper. However, basic features of the code must be outlined according to Item 1, above.
- (9)
Comparison to appropriate analytical or well-established numerical benchmark solutions may be used to demonstrate accuracy for another class of problems. However, in general this does not demonstrate accuracy for another class of problems, especially if any adjustable parameters are involved, as in turbulence modelling.
- (10)
Comparison with reliable experimental results is appropriate, provided experimental uncertainty is established. However, “reasonable agreement” with experimental data alone will not be enough to justify a given singlegrid calculation, especially if adjustable parameters are involved.
These ten items lay down a set of criteria by which the editors and reviewers of this Journal will judge the archival quality of publications dealing with computational studies for the Journal of Fluids Engineering. We recognize that the effort to perform a thorough study of numerical accuracy may be great and that many practical engineering calculations will continue to be performed by first order methods, on a single fixed grid. However, such analyses would not be appropriate for presentation in this archival journal. With the gains in performance of low-end workstations, it is now reasonable to require papers on solutions by CFD to meet these fundamental criteria for archiving of a publication.
With the details of these ten criteria now presented, a shortened statement will appear in each volume of the journal. This statement will appear as follows:
The Journal of Fluids Engineering will not consider any paper reporting the numerical solution of a fluids engineering problem that fails to address the task of systematic truncation error testing and accuracy estimation. Authors should address the following criteria for assessing numerical uncertainty.
- (1)
The basic features of the method including formal truncation error of individual terms in the governing numerical equations must be described.
- (2)
Methods must be at least second order accurate in space.
- (3)
Inherent or explicit artificial viscosity (or diffusivity) must be assessed and minimized.
- (4)
Grid independence or convergence must be established.
- (5)
When appropriate, iterative convergence must be addressed.
- (6)
In transient calculations, phase error must be assessed and minimized.
- (7)
The accuracy and implementation of boundary and initial conditions must be fully explained.
- (8)
An existing code must be fully cited in easily available references.
- (9)
Benchmark solutions may be used for validation for a specific class of problems.
- (10)
Reliable experimental results may be used to validate a solution.
