Parameter estimates in large-scale complex engineered systems (LaCES) affect system evolution, yet can be difficult and expensive to test. Systems engineering uses analytical methods to reduce uncertainty, but a growing body of work from other disciplines indicates that cognitive heuristics also affect decision-making. Results from interviews with expert aerospace practitioners suggest that engineers bias estimation strategies. Practitioners reaffirmed known system features and posited that engineers may bias estimation methods as a negotiation and resource conservation strategy. Specifically, participants reported that some systems engineers “game the system” by biasing requirements to counteract subsystem estimation biases. An agent-based model (ABM) simulation which recreates these characteristics is presented. Model results suggest that system-level estimate accuracy and uncertainty depend on subsystem behavior and are not significantly affected by systems engineers' “gaming” strategy.

References

References
1.
Bloebaum
,
C. L.
, and
McGowan
,
A.-M. R.
,
2012
, “
The Design of Large-Scale Complex Engineered Systems: Present Challenges and Future Promise
,”
AIAA
Paper No. 2012-5571.
2.
McGowan
,
A.-M. R.
,
Daly
,
S.
,
Baker
,
W.
,
Papalambros
,
P.
, and
Seifert
,
C.
,
2013
, “
A Socio-Technical Perspective on Interdisciplinary Interactions During the Development of Complex Engineered Systems
,”
Procedia Comput. Sci.
,
16
, pp.
1142
1151
.
3.
McGowan
,
A.-M. R.
,
2014
, “
Interdisciplinary Interactions During R & D and Early Design of Large Engineered Systems
,”
Ph.D. thesis
, University of Michigan, Ann Arbor, MI.https://ntrs.nasa.gov/archive/nasa/casi.ntrs.nasa.gov/20140004579.pdf
4.
Yassine
,
A.
, and
Braha
,
D.
,
2003
, “
Complex Concurrent Engineering and the Design Structure Matrix Method
,”
Concurrent Eng. Res. Appl.
,
11
(
3
), pp.
165
176
.
5.
Eppinger
,
S.
,
1991
, “
Model-Based Approaches to Managing Concurrent Engineering
,”
J. Eng. Des.
,
2
(
4
), pp.
283
290
.
6.
Ye, Y.
,
Jankovic, M.
, and
Kremer, G. E.
, 2015, “
Understanding the Impact of Subjective Uncertainty on Architecture and Supplier Identification in Early Complex Systems Design
,”
ASME J. Risk Uncertainty Part B
,
1
(
3
), p.
031005
.
7.
de Weck, O. L.
,
Eckert, C. M.
, and
Clarkson, J.
,
2007
, “
A Classification of Uncertainty for Early Product and System Design
,”
Guidelines for a Decision Support Method Adapted to NPD Processes
, Massachusetts Institute of Technology, Cambridge, MA.
8.
Crossland
,
R.
,
Williams
,
J. H. S.
, and
McMahon
,
C. A.
,
2003
, “
An Object-Oriented Modeling Framework for Representing Uncertainty in Early Variant Design
,”
Res. Eng. Des.
,
14
(
3
), pp.
173
183
.
9.
Dubos
,
G. F.
,
Castet
,
J. F.
, and
Saleh
,
J. H.
,
2010
, “
Statistical Reliability Analysis of Satellites by Mass Category: Does Spacecraft Size Matter?
Acta Astronaut.
,
67
(
5–6
), pp.
584
595
.
10.
Ryan
,
R. S.
, and
Townsend
,
J. S.
,
1997
, “
Fundamentals and Issues in Launch Vehicle Design
,”
J. Spacecr. Rockets
,
34
(
2
), pp.
192
198
.
11.
Chang
,
I.-S.
,
1996
, “
Investigation of Space Launch Vehicle Catastrophic Failures
,”
J. Spacecr. Rockets
,
33
(
2
), pp.
198
205
.
12.
Kapurch
,
S. J.
,
2007
,
NASA Systems Engineering Handbook
,
The National Aeronautics and Space Administration
, Diane Publishing, Collingsdale, PA, p.
360
.
13.
Cardin
,
M.-A.
,
Nuttall
,
W. J.
,
De Neufville
,
R.
, and
Dahlgren
,
J.
,
2007
, “
Extracting Value From Uncertainty: A Methodology for Engineering Systems Design
,”
17th Symposium International Council on System Engineering
, San Diego, CA, June 24–28, pp.
1
9
.
14.
Forsberg
,
K.
, and
Mooz
,
H.
,
1994
, “
The Relationship of System Engineering to the Project Cycle
,”
12th INTERNET World Congress Project Management
, Oslo, Norway, June 9–11, p. 12.
15.
Keating
,
C.
,
Rogers
,
R.
,
Unal
,
R.
,
Dryer
,
D.
,
Sousa-Poza
,
A.
,
Safford
,
R.
,
Peterson
,
W.
, and
Rabadi
,
G.
,
2008
, System of Systems Engineering,
IEEE Eng. Manage. Rev.
,
36
(4), p. 62.
16.
Clark
,
J. O.
,
2009
, “
System of Systems Engineering and Family of Systems Engineering From a Standards, V-Model, and Dual-V Model Perspective
,” Third Annual
IEEE
System Conference, Vancouver, BC, Canada, Mar. 23–26, pp.
381
387
.
17.
Collopy
,
P. D.
, and
Hollingsworth
,
P. M.
,
2011
, “
Value-Driven Design
,”
J. Aircr.
,
48
(
3
), pp.
749
759
.
18.
Collopy
,
P. D.
,
Bloebaum
,
C. L.
, and
Mesmer
,
B. L.
,
2012
, “
The Distinct and Interrelated Roles of Value-Driven Design, Multidisciplinary Design Optimization, and Decision Analysis
,”
AIAA
Paper No. 2012-5575.
19.
Mullan
,
C.
,
Price
,
M.
,
Soban
,
D.
, and
Fanthorpe
,
C.
,
2012
, “
Development of Subsystem Relationships Using a Value Driven Design Framework
,”
AIAA
Paper No. 2012-5459.
20.
Bertoni
,
M.
,
Bertoni
,
A.
, and
Isaksson
,
O.
,
2016
, “
Evoke: A Value-Driven Concept Selection Method for Early System Design
,”
J. Syst. Sci. Syst. Eng.
,
27
(1), pp.
46
77
.
21.
Kahneman
,
D.
,
Slovic
,
P.
, and
Tversky
,
A.
,
1982
,
Judgment Under Uncertainty: Heuristics and Biases
,
Cambridge University Press
, Cambridge, UK.
22.
Slovic
,
P.
,
Finucane
,
M.
,
Peters
,
E.
, and
MacGregor
,
D. G.
,
2002
, “
Rational Actors or Rational Fools: Implications of the Affect Heuristic for Behavioral Economics
,”
J. Socio. Econ.
,
31
(
4
), pp.
329
342
.
23.
Wertz
,
J. R.
,
Everett
,
D. F.
, and
Puschell
,
J. J.
,
2011
,
Space Mission Engineering: the New SMAD
,
Microcosm Press
, Portland, OR.
24.
Collopy
,
P. D.
,
2012
, “
A Research Agenda for the Coming Renaissance
,”
AIAA
Paper No. 2012-799.
25.
Kim
,
H. M.
,
Michelena
,
N. F.
,
Papalambros
,
P. Y.
, and
Jiang
,
T.
,
2003
, “
Target Cascading in Optimal System Design
,”
ASME J. Mech. Des.
,
125
(
3
), p.
474
.
26.
Tibor
,
E. B.
,
Mesmer
,
B. L.
,
Bloebaum
,
C. L.
,
Miller
,
S. W.
, and
Simpson
,
T. W.
,
2014
, “
Visualization of System Decomposition in a Value-Based Framework
,”
AIAA
Paper No. 2014-2181.
27.
Kannan
,
H.
,
Tibor
,
E.
,
Mesmer
,
B.
, and
Bloebaum
,
C. L.
,
2015
, “
Incorporation of Coupling Strength Models in a Value-Based Systems Engineering Framework for Optimization
,”
AIAA
Paper No. 2015-3086.
28.
Kwasa
,
B.
,
Bloebaum
,
C. L.
, and
Mesmer
,
B.
,
2015
, “
Value Impact of an Organization Structure in the Context of Value-Driven Design
,”
AIAA
Paper No. 2015-1364.
29.
Bhatia
,
G. V.
,
Kannan
,
H.
, and
Bloebaum
,
C. L.
,
2016
, “
A Game Theory Approach to Bargaining Over Attributes of Complex Systems in the Context of Value-Driven Design
,”
AIAA
Paper No. 2016-0972.
30.
Kannan
,
H.
,
Mesmer
,
B.
, and
Bloebaum
,
C. L.
,
2016
, “
Incorporation of Risk Preferences in a Value-Based Systems Engineering Framework for a Satellite System
,”
AIAA
Paper No. 2016-0681.
31.
Subramanian
,
T. R.
,
Khol
,
A. R.
,
Kannan
,
H.
,
Winer
,
E. H.
,
Bloebaum
,
C. L.
, and
Mesmer
,
B.
,
2016
, “
Understanding the Impact of Uncertainty on the Fidelity of the Value Model
,”
AIAA
Paper No. 2016-0973.
32.
Herrmann
,
J. W.
,
2015
,
Engineering Decision Making and Risk Management
,
John Wiley & Sons
, Hoboken, NJ.
33.
Austin-Breneman
,
J.
,
Yu
,
B. Y.
, and
Yang
,
M. C.
,
2015
, “
Biased Information Passing Between Subsystems Over Time in Complex System Design
,”
ASME J. Mech. Des.
,
138
(
1
), p.
011101
.
34.
Takamatsu
,
T.
,
Hashimoto
,
I.
, and
Ohno
,
H.
,
1970
, “
Optimal Design of a Large Complex System From the Viewpoint of Sensitivity Analysis
,”
Ind. Eng. Chem. Process Des. Dev.
,
9
(
3
), pp.
368
379
.
35.
Thunnisen
,
D. P.
,
2004
, “
Method for Determining Margins in Conceptual Design
,”
J. Spacecr. Rockets
,
41
(
1
), pp.
85
92
.
36.
Gu
,
X.
,
Renaud
,
J.
,
Batill
,
S.
,
Brach
,
R.
, and
Budhiraja
,
A.
,
2000
, “
Worst Case Propagated Uncertainty of Multidisciplinary Systems in Robust Design Optimization
,”
Struct. Multidiscip. Optim.
,
20
(
3
), pp.
190
213
.
37.
Helton
,
J. C.
,
2011
, “
Quantification of Margins and Uncertainties: Conceptual and Computational Basis
,”
Reliab. Eng. Syst. Saf.
,
96
(
9
), pp.
976
1013
.
38.
Sentz
,
K.
, and
Ferson
,
S.
,
2011
, “
Probabilistic Bounding Analysis in the Quantification of Margins and Uncertainties
,”
Reliab. Eng. Syst. Saf.
,
96
(
9
), pp.
1126
1136
.
39.
Anderson
,
N. H.
, and
Barrios
,
A. A.
,
1961
, “
Primacy Effects in Personality Impression Formation
,”
J. Abnorm. Soc. Psychol.
,
63
(
2
), pp.
346
350
.
40.
Wallsten
,
T. S.
,
1981
, “
Physician and Medical Student Bias in Evaluating Diagnostic Information
,”
Med. Decis. Making
,
1
(
2
), pp.
145
64
.
41.
Wallsten
,
T. S.
,
1983
, “
The Theoretical Status of Judgmental Heuristics
,”
Adv. Psychol.
,
16
, pp.
21
37
.
42.
Slovic
,
P.
, and
Lichtenstein
,
S.
,
1971
, “
Comparison of Bayesian and Regression Approaches to the Study of Information Processing in Judgment
,”
Organ. Behav. Hum. Perform.
,
6
(
6
), pp.
649
744
.
43.
Finucane
,
M. L.
,
Alhakami
,
A.
,
Slovic
,
P.
, and
Johnson
,
S. M.
,
2000
, “
The Affect Heuristic in Judgments of Risks and Benefits
,”
J. Behav. Decis. Mak.
,
13
(
1
), pp.
1
17
.
44.
Macy
,
M. W.
, and
Willer
,
R.
,
2002
, “
From Factors to Factors: Computational Sociology and Agent-Based Modeling
,”
Annu. Rev. Sociol.
,
28
(
1
), pp.
143
166
.
45.
Macal, C. M.
, and
North, M. J
.,
2008
, “
Agent-Based Modeling and Simulation
,”
Proceedings of the 40th Conference on Winter Simulation
, Miami, FL, Dec. 7–10, pp.
101
112
.
46.
McComb
,
C.
,
Cagan
,
J.
, and
Kotovsky
,
K.
,
2016
, “
Drawing Inspiration From Human Design Teams For Better Search And Optimization: The Heterogeneous Simulated Annealing Teams Algorithm
,”
ASME J. Mech. Des.
,
138
(
4
), p.
044501
.
47.
Triki
,
E.
,
Collette
,
Y.
, and
Siarry
,
P.
,
2005
, “
A Theoretical Study on the Behavior of Simulated Annealing Leading to a New Cooling Schedule
,”
Eur. J. Oper. Res.
,
166
(
1
), pp.
77
92
.
48.
Patton
,
M. Q.
,
2002
,
Qualitative Research and Evaluation Methods
,
SAGE
, Thousand Oaks, CA.
49.
Anderson
,
J. C.
, and
Rungtusanatham
,
M.
,
1994
, “
A Theory of Quality Management Underlying the Deming Management Method
,”
Acad. Manag. Rev.
,
19
(
3
), pp.
472
509
.
50.
Azzalini
,
A.
,
1985
, “
A Class of Distributions Which Includes the Normal Ones
,”
Scand. J. Stat.
,
12
(
2
), pp.
171
178
.https://www.jstor.org/stable/4615982
51.
Glaser
,
B. G.
, and
Strauss
,
A. L.
,
1967
,
The Discovery of Grounded Theory: Strategies for Qualitative Research
, Aldine Transactions, New Brunswick, NJ.
52.
Corbin
,
J. M.
, and
Strauss
,
A.
,
2008
,
Basics of Qualitative Research: Techniques and Procedures for Developing Grounded Theory
,
SAGE
, Thousand Oaks, CA.
You do not currently have access to this content.