Objective evaluation (OE) methods provide quantitative insight into how well time history data from computational models match data from physical systems. Two feature specific techniques commonly used for this purpose are cora and the ISO/TS 18571 standards. These ostensibly objective techniques have differences in their algorithms that lead to discrepancies when interpreting their results. The objectives of this study were (1) to apply both techniques to a dataset from a computational model, and compare the scores and (2) conduct a survey of subject matter experts (SMEs) to determine which OE method compares more consistently with SME interpretation. The GHBMC male human model was used in simulations of biomechanics experiments, producing 58 time history curves. Because both techniques produce scores based on specific features of the signal comparison (phase, size, and shape), 174 pairwise comparisons were made. Statistical analysis revealed significant differences between the two OE methods for each component rating metric. SMEs (n = 40) surveyed scored how well the computational traces matched the experiments for the three rating metrics. SME interpretation was found to statistically agree with the ISO shape and phase metrics, but was significantly different from the ISO size rating. SME interpretation agreed with the cora size rating. The findings suggest that when possible, engineers should use a mixed approach to reporting objective ratings, using the ISO shape and phase methods, and size methods of cora. We recommend to weight metrics greatest to least for shape, phase, and size. Given the general levels of agreement observed and the sample size, the results require a nuanced interpretation.

References

References
1.
Tabiei
,
A.
, and
Wu
,
J.
,
2000
, “
Roadmap for Crashworthiness Finite Element Simulation of Roadside Safety Structures
,”
Finite Elem. Anal. Des.
,
34
(
2
), pp.
145
157
.
2.
Davis
,
M. L.
,
Vavalle
,
N. A.
, and
Gayzik
,
F. S.
,
2015
, “
An Evaluation of Mass-Normalization Using 50th and 95th Percentile Human Body Finite Element Models in Frontal Crash
,”
International Research Council on Biomechanics of Injury
(
IRCOBI
),
Lyon, France
,
Sept. 9–11
, pp.
608
621
.http://www.ircobi.org/wordpress/downloads/irc15/pdf_files/68.pdf
3.
Untaroiu
,
C. D.
,
Crandall
,
J. R.
,
Takahashi
,
Y.
,
Okamoto
,
M.
,
Ito
,
O.
, and
Fredriksson
,
R.
,
2010
, “
Analysis of Running Child Pedestrians Impacted by a Vehicle Using Rigid-Body Models and Optimization Techniques
,”
Saf. Sci.
,
48
(
2
), pp.
259
267
.
4.
Gaewsky
,
J. P.
,
Weaver
,
A. A.
,
Koya
,
B.
, and
Stitzel
,
J. D.
,
2015
, “
Driver Injury Risk Variability in Finite Element Reconstructions of Crash Injury Research and Engineering Network (CIREN) Frontal Motor Vehicle Crashes
,”
Traffic Inj. Prev.
,
16
(
Suppl. 2
), pp.
S124
S131
.
5.
Vavalle
,
N. A.
,
Moreno
,
D. P.
,
Rhyne
,
A. C.
,
Stitzel
,
J. D.
, and
Gayzik
,
F. S.
,
2013
, “
Lateral Impact Validation of a Geometrically Accurate Full Body Finite Element Model for Blunt Injury Prediction
,”
Ann. Biomed. Eng.
,
41
(
3
), pp.
497
512
.
6.
Vavalle
,
N. A.
,
2012
, “
Validation of the Global Human Body Models Consortium Mid-Sized Male Model in Lateral Impacts and Sled Tests
,” Wake Forest University, Winston-Salem, NC.
7.
Sprague
,
M. A.
, and
Geers
,
T. L.
,
2004
, “
A Spectral-Element Method for Modelling Cavitation in Transient Fluid-Structure Interaction
,”
Int. J. Numer. Methods Eng.
,
60
(
15
), pp.
2467
2499
.
8.
Rhule
,
H. H.
,
Maltese
,
M. R.
,
Donnelly
,
B. R.
,
Eppinger
,
R. H.
,
Brunner
,
J. K.
, and
Bolte
,
J. H.
,
2002
, “
Development of a New Biofidelity Ranking System for Anthropomorphic Test Devices
,”
Stapp Car Crash J.
,
46
, pp.
477
512
.https://www.researchgate.net/publication/6701651_Development_of_a_new_biofidelity_ranking_system_for_anthropomorphic_test_devices
9.
Vavalle
,
N. A.
,
Jelen
,
B. C.
,
Moreno
,
D. P.
,
Stitzel
,
J. D.
, and
Gayzik
,
F. S.
,
2013
, “
An Evaluation of Objective Rating Methods for Full-Body Finite Element Model Comparison to PMHS Tests
,”
Traffic Inj. Prev.
,
14
(
Suppl. 1
), pp.
S87
S94
.
10.
Gehre
,
C.
,
Gades
,
H.
, and
Wernicke
,
P.
, 2009, “
Objective Rating of Signals Using Test and Simulation Responses
,”
21st ESV Conference
,
Stuttgart, Germany
,
June 15–19
, Paper No. 09-0407-O.https://trid.trb.org/view/1100058
11.
Barbat
,
S.
,
Fu
,
Y.
,
Zhan
,
Z.
,
Yang
,
R. J.
, and
Gehre
,
C.
,
2013
, “
Objective Rating Metric for Dynamic Systems
,” Enhanced Safety of Vehicles, Seoul, South Korea, May 27–30, Paper No. 13-0448-O.
12.
Poulard
,
D.
,
Subit
,
D.
,
Nie
,
B.
,
Donlon
,
J.-P.
, and
Kent
,
R. W.
,
2015
, “
The Contribution of Pre-Impact Posture on Restrained Occupant Finite Element Model Response in Frontal Impact
,”
Traffic Inj. Prev.
,
16
(
Suppl. 2
), pp.
S87
S95
.
13.
Gehre
,
C.
, and
Stahlschmidt
,
S.
, 2011, “
Assessment of Dummy Models by Using Objective Rating Methods
,”
22nd International Technical Conference on the Enhanced Safety of Vehicles
,
Washington, DC
,
June 13–16
, Paper No. 11-0216.
14.
Schwer
,
L. E.
,
2007
, “
Validation Metrics for Response Histories: Perspectives and Case Studies
,”
Eng. Comput.
,
23
(
4
), pp.
295
309
.
15.
Teferra
,
K.
,
Shields
,
M. D.
,
Hapij
,
A.
, and
Daddazio
,
R. P.
,
2014
, “
Mapping Model Validation Metrics to Subject Matter Expert Scores for Model Adequacy Assessment
,”
Reliab. Eng. Syst. Saf.
,
132
, pp.
9
19
.
16.
Gayzik
,
F. S.
,
Moreno
,
D. M.
,
Geer
,
C. P.
,
Wuertzer
,
S. D.
,
Martin
,
R. S.
, and
Stitzel
,
J. D.
,
2011
, “
Development of a Full Body CAD Dataset for Computational Modeling: A Multi-Modality Approach
,”
Ann. Biomed. Eng.
,
39
(
10
), pp.
2568
2583
.
17.
Li
,
Z.
,
Kindig
,
M. W.
,
Kerrigan
,
J. R.
,
Untaroiu
,
C. D.
,
Subit
,
D.
,
Crandall
,
J. R.
, and
Kent
,
R. W.
,
2010
, “
Rib Fractures Under Anterior-Posterior Dynamic Loads: Experimental and Finite-Element Study
,”
J. Mech.
,
43
(
2
), pp.
228
234
.
18.
Soni
,
A.
, and
Beillas
,
P.
,
2015
, “
Modelling Hollow Organs for Impact Conditions: A Simplified Case Study
,”
Comput. Methods Biomech. Biomed. Eng.
,
18
(
7
), pp.
730
739
.
19.
Shin
,
J.
,
Yue
,
N.
, and
Untaroiu
,
C. D.
,
2012
, “
A Finite Element Model of the Foot and Ankle for Automotive Impact Applications
,”
Ann. Biomed. Eng.
,
40
(
12
), pp.
2519
2531
.
20.
DeWit
,
J. A.
, and
Cronin
,
D. S.
,
2012
, “
Cervical Spine Segment Finite Element Model for Traumatic Injury Prediction
,”
J. Mech. Behav. Biomed. Mater.
,
10
, pp.
138
150
.
21.
Yang
,
K. H.
,
Hu
,
J.
,
White
,
N. A.
,
King
,
A. I.
,
Chou
,
C. C.
, and
Prasad
,
P.
,
2006
, “
Development of Numerical Models for Injury Biomechanics Research: A Review of 50 Years of Publications in the Stapp Car Crash Conference
,”
Stapp Car Crash J.
,
50
, pp.
429
490
.https://wwww.unboundmedicine.com/medline/citation/17311173/Development_of_numerical_models_for_injury_biomechanics_research:_a_review_of_50_years_of_publications_in_the_Stapp_Car_Crash_Conference_
22.
Toyota
,
2010
, “
Documentation of Total Human Model for Safety (THUMS) AM50 Pedestrian/Occupant Model
,” Toyota Motor Corporation, Nagakute, Japan.
23.
Hayes
,
A. R.
,
Vavalle
,
N. A.
,
Moreno
,
D. P.
,
Stitzel
,
J. D.
, and
Gayzik
,
F. S.
,
2014
, “
Validation of Simulated Chestband Data in Frontal and Lateral Loading Using a Human Body Finite Element Model
,”
Traffic Inj. Prev.
,
15
(
2
), pp.
181
186
.
24.
Viano
,
D. C.
,
1989
, “
Biomechanical Responses and Injuries in Blunt Lateral Impact
,”
Stapp Car Crash J.
,
33
, pp.
113
142
.
25.
Hardy
,
W. N.
,
Schneider
,
L. W.
, and
Rouhana
,
S. W.
,
2001
, “
Abdominal Impact Response to Rigid-Bar, Seatbelt, and Airbag Loading
,”
Stapp Car Crash J.
,
45
, pp.
1
32
.https://www.researchgate.net/publication/6369933_Abdominal_impact_response_to_rigid-bar_seatbelt_and_airbag_loading
26.
Bouquet
,
R.
,
Ramet
,
M.
,
Bermond
,
F.
,
Caire
,
Y.
,
Talantikite
,
Y.
,
Robin
,
S.
, and
Voiglio
,
E.
, 1998 “
Pelvis Human Response to Lateral Impact
,”
16th International Technical Conference on the Enhanced Safety of Vehicles
,
Windsor, ON, Canada
, pp.
1665
1686
.
27.
Cavanaugh
,
J. M.
,
Walilko
,
T. J.
,
Malhotra
,
A.
,
Zhu
,
Y.
, and
King
,
A. I.
,
1990
, “
Biomechanical Response and Injury Tolerance of the Pelvis in Twelve Sled Side Impacts
,”
SAE Paper No. 902305
.
28.
Cavanaugh
,
J. M.
,
Zhu
,
Y.
,
Huang
,
Y.
, and
King
,
A. I.
,
1993
, “
Injury and Response of the Thorax in Side Impact Cadaveric Tests
,”
SAE
Paper No. 933127
.
29.
Shaw
,
G.
,
Parent
,
D.
,
Purtsezov
,
S.
,
Lessley
,
D.
,
Crandall
,
J.
,
Kent
,
R.
,
Guillemot
,
H.
,
Ridella
,
S. A.
,
Takhounts
,
E.
, and
Martin
,
P.
,
2009
, “
Impact Response of Restrained PMHS in Frontal Sled Tests: Skeletal Deformation Patterns Under Seat Belt Loading
,”
Stapp Car Crash J.
,
53
, pp.
1
48
.https://pdfs.semanticscholar.org/6bb3/a6ec351b40fac200129692dc29cb2e6b6057.pdf
30.
Thunert
,
C.
,
2012
, “
CORA Release 3.6 User's Manual
,” GNS, Ingolstadt, Germany.
31.
ISO/IOFS.
,
2013
, “
Road Vehicles—Objective Rating Metric for Non-Ambiguous Signals
,” International Organization for Standardization, Geneva, Switzerland, Standard No.
ISO/TS 18571
.https://www.iso.org/standard/62937.html
32.
Sarin
,
H.
,
Barbat
,
S.
,
Yang
,
R. J.
,
Kokkolaras
,
M.
,
Hulbert
,
G.
, and
Papalambros
,
P.
,
2010
, “
Comparing Time Histories for Validation of Simulation Models: Error Measures and Metrics
,”
ASME J. Dyn. Syst. Meas. Control
,
132
(
6
), p.
061401
.
33.
Vavalle
,
N. A.
,
Davis
,
M. L.
,
Stitzel
,
J. D.
, and
Gayzik
,
F. S.
,
2015
, “
Quantitative Validation of a Human Body Finite Element Model Using Rigid Body Impacts
,”
Ann. Biomed. Eng.
,
43
(
9
), pp.
2163
2174
.
34.
Davis
,
M. L.
,
Koya
,
B.
,
Schap
,
J. M.
, and
Gayzik
,
F. S.
,
2016
, “
Development and Full Body Validation of a 5th Percentile Female Finite Element Model
,”
Stapp Car Crash J.
,
60
, pp.
509
544
.https://search.proquest.com/openview/940c863f1e7b068c369c97c2bb6e1acb/1.pdf?pq-origsite=gscholar&cbl=29197
You do not currently have access to this content.