Parameter estimates in large-scale complex engineered systems (LaCES) affect system evolution, yet can be difficult and expensive to test. Systems engineering uses analytical methods to reduce uncertainty, but a growing body of work from other disciplines indicates that cognitive heuristics also affect decision-making. Results from interviews with expert aerospace practitioners suggest that engineers bias estimation strategies. Practitioners reaffirmed known system features and posited that engineers may bias estimation methods as a negotiation and resource conservation strategy. Specifically, participants reported that some systems engineers “game the system” by biasing requirements to counteract subsystem estimation biases. An agent-based model (ABM) simulation which recreates these characteristics is presented. Model results suggest that system-level estimate accuracy and uncertainty depend on subsystem behavior and are not significantly affected by systems engineers' “gaming” strategy.
Issue Section:
Design Theory and Methodology
References
References
1.
Bloebaum
, C. L.
McGowan
, A.-M. R.
2012
, “The Design of Large-Scale Complex Engineered Systems: Present Challenges and Future Promise
,” AIAA
Paper No. 2012-5571.2.
McGowan
, A.-M. R.
Daly
, S.
Baker
, W.
Papalambros
, P.
Seifert
, C.
2013
, “A Socio-Technical Perspective on Interdisciplinary Interactions During the Development of Complex Engineered Systems
,” Procedia Comput. Sci.
, 16
, pp. 1142
–1151
.3.
McGowan
, A.-M. R.
2014
, “Interdisciplinary Interactions During R & D and Early Design of Large Engineered Systems
,” Ph.D. thesis
, University of Michigan, Ann Arbor, MI.https://ntrs.nasa.gov/archive/nasa/casi.ntrs.nasa.gov/20140004579.pdf4.
Yassine
, A.
Braha
, D.
2003
, “Complex Concurrent Engineering and the Design Structure Matrix Method
,” Concurrent Eng. Res. Appl.
, 11
(3
), pp. 165
–176
.5.
Eppinger
, S.
1991
, “Model-Based Approaches to Managing Concurrent Engineering
,” J. Eng. Des.
, 2
(4
), pp. 283
–290
.6.
Ye, Y.
, Jankovic, M.
, and Kremer, G. E.
, 2015, “Understanding the Impact of Subjective Uncertainty on Architecture and Supplier Identification in Early Complex Systems Design
,” ASME J. Risk Uncertainty Part B
, 1
(3
), p. 031005
.7.
de Weck, O. L.
, Eckert, C. M.
, and Clarkson, J.
, 2007
, “A Classification of Uncertainty for Early Product and System Design
,” Guidelines for a Decision Support Method Adapted to NPD Processes
, Massachusetts Institute of Technology, Cambridge, MA.8.
Crossland
, R.
Williams
, J. H. S.
McMahon
, C. A.
2003
, “An Object-Oriented Modeling Framework for Representing Uncertainty in Early Variant Design
,” Res. Eng. Des.
, 14
(3
), pp. 173
–183
.9.
Dubos
, G. F.
Castet
, J. F.
Saleh
, J. H.
2010
, “Statistical Reliability Analysis of Satellites by Mass Category: Does Spacecraft Size Matter?
” Acta Astronaut.
, 67
(5–6
), pp. 584
–595
.10.
Ryan
, R. S.
Townsend
, J. S.
1997
, “Fundamentals and Issues in Launch Vehicle Design
,” J. Spacecr. Rockets
, 34
(2
), pp. 192
–198
.11.
Chang
, I.-S.
1996
, “Investigation of Space Launch Vehicle Catastrophic Failures
,” J. Spacecr. Rockets
, 33
(2
), pp. 198
–205
.12.
Kapurch
, S. J.
2007
, NASA Systems Engineering Handbook
, The National Aeronautics and Space Administration
, Diane Publishing, Collingsdale, PA, p. 360
.13.
Cardin
, M.-A.
Nuttall
, W. J.
De Neufville
, R.
Dahlgren
, J.
2007
, “Extracting Value From Uncertainty: A Methodology for Engineering Systems Design
,” 17th Symposium International Council on System Engineering
, San Diego, CA, June 24–28, pp. 1
–9
.14.
Forsberg
, K.
Mooz
, H.
1994
, “The Relationship of System Engineering to the Project Cycle
,” 12th INTERNET World Congress Project Management
, Oslo, Norway, June 9–11, p. 12.15.
Keating
, C.
Rogers
, R.
Unal
, R.
Dryer
, D.
Sousa-Poza
, A.
Safford
, R.
Peterson
, W.
Rabadi
, G.
2008
, System of Systems Engineering, IEEE Eng. Manage. Rev.
, 36
(4), p. 62.16.
Clark
, J. O.
2009
, “System of Systems Engineering and Family of Systems Engineering From a Standards, V-Model, and Dual-V Model Perspective
,” Third Annual IEEE
System Conference, Vancouver, BC, Canada, Mar. 23–26, pp. 381
–387
.17.
Collopy
, P. D.
Hollingsworth
, P. M.
2011
, “Value-Driven Design
,” J. Aircr.
, 48
(3
), pp. 749
–759
.18.
Collopy
, P. D.
Bloebaum
, C. L.
Mesmer
, B. L.
2012
, “The Distinct and Interrelated Roles of Value-Driven Design, Multidisciplinary Design Optimization, and Decision Analysis
,” AIAA
Paper No. 2012-5575.19.
Mullan
, C.
Price
, M.
Soban
, D.
Fanthorpe
, C.
2012
, “Development of Subsystem Relationships Using a Value Driven Design Framework
,” AIAA
Paper No. 2012-5459.20.
Bertoni
, M.
Bertoni
, A.
Isaksson
, O.
2016
, “Evoke: A Value-Driven Concept Selection Method for Early System Design
,” J. Syst. Sci. Syst. Eng.
, 27
(1), pp. 46
–77
.21.
Kahneman
, D.
Slovic
, P.
Tversky
, A.
1982
, Judgment Under Uncertainty: Heuristics and Biases
, Cambridge University Press
, Cambridge, UK.22.
Slovic
, P.
Finucane
, M.
Peters
, E.
MacGregor
, D. G.
2002
, “Rational Actors or Rational Fools: Implications of the Affect Heuristic for Behavioral Economics
,” J. Socio. Econ.
, 31
(4
), pp. 329
–342
.23.
Wertz
, J. R.
Everett
, D. F.
Puschell
, J. J.
2011
, Space Mission Engineering: the New SMAD
, Microcosm Press
, Portland, OR.24.
Collopy
, P. D.
2012
, “A Research Agenda for the Coming Renaissance
,” AIAA
Paper No. 2012-799.25.
Kim
, H. M.
Michelena
, N. F.
Papalambros
, P. Y.
Jiang
, T.
2003
, “Target Cascading in Optimal System Design
,” ASME J. Mech. Des.
, 125
(3
), p. 474
.26.
Tibor
, E. B.
Mesmer
, B. L.
Bloebaum
, C. L.
Miller
, S. W.
Simpson
, T. W.
2014
, “Visualization of System Decomposition in a Value-Based Framework
,” AIAA
Paper No. 2014-2181.27.
Kannan
, H.
Tibor
, E.
Mesmer
, B.
Bloebaum
, C. L.
2015
, “Incorporation of Coupling Strength Models in a Value-Based Systems Engineering Framework for Optimization
,” AIAA
Paper No. 2015-3086.28.
Kwasa
, B.
Bloebaum
, C. L.
Mesmer
, B.
2015
, “Value Impact of an Organization Structure in the Context of Value-Driven Design
,” AIAA
Paper No. 2015-1364.29.
Bhatia
, G. V.
Kannan
, H.
Bloebaum
, C. L.
2016
, “A Game Theory Approach to Bargaining Over Attributes of Complex Systems in the Context of Value-Driven Design
,” AIAA
Paper No. 2016-0972.30.
Kannan
, H.
Mesmer
, B.
Bloebaum
, C. L.
2016
, “Incorporation of Risk Preferences in a Value-Based Systems Engineering Framework for a Satellite System
,” AIAA
Paper No. 2016-0681.31.
Subramanian
, T. R.
Khol
, A. R.
Kannan
, H.
Winer
, E. H.
Bloebaum
, C. L.
Mesmer
, B.
2016
, “Understanding the Impact of Uncertainty on the Fidelity of the Value Model
,” AIAA
Paper No. 2016-0973.32.
Herrmann
, J. W.
2015
, Engineering Decision Making and Risk Management
, John Wiley & Sons
, Hoboken, NJ.33.
Austin-Breneman
, J.
Yu
, B. Y.
Yang
, M. C.
2015
, “Biased Information Passing Between Subsystems Over Time in Complex System Design
,” ASME J. Mech. Des.
, 138
(1
), p. 011101
.34.
Takamatsu
, T.
Hashimoto
, I.
Ohno
, H.
1970
, “Optimal Design of a Large Complex System From the Viewpoint of Sensitivity Analysis
,” Ind. Eng. Chem. Process Des. Dev.
, 9
(3
), pp. 368
–379
.35.
Thunnisen
, D. P.
2004
, “Method for Determining Margins in Conceptual Design
,” J. Spacecr. Rockets
, 41
(1
), pp. 85
–92
.36.
Gu
, X.
Renaud
, J.
Batill
, S.
Brach
, R.
Budhiraja
, A.
2000
, “Worst Case Propagated Uncertainty of Multidisciplinary Systems in Robust Design Optimization
,” Struct. Multidiscip. Optim.
, 20
(3
), pp. 190
–213
.37.
Helton
, J. C.
2011
, “Quantification of Margins and Uncertainties: Conceptual and Computational Basis
,” Reliab. Eng. Syst. Saf.
, 96
(9
), pp. 976
–1013
.38.
Sentz
, K.
Ferson
, S.
2011
, “Probabilistic Bounding Analysis in the Quantification of Margins and Uncertainties
,” Reliab. Eng. Syst. Saf.
, 96
(9
), pp. 1126
–1136
.39.
Anderson
, N. H.
Barrios
, A. A.
1961
, “Primacy Effects in Personality Impression Formation
,” J. Abnorm. Soc. Psychol.
, 63
(2
), pp. 346
–350
.40.
Wallsten
, T. S.
1981
, “Physician and Medical Student Bias in Evaluating Diagnostic Information
,” Med. Decis. Making
, 1
(2
), pp. 145
–64
.41.
Wallsten
, T. S.
1983
, “The Theoretical Status of Judgmental Heuristics
,” Adv. Psychol.
, 16
, pp. 21
–37
.42.
Slovic
, P.
Lichtenstein
, S.
1971
, “Comparison of Bayesian and Regression Approaches to the Study of Information Processing in Judgment
,” Organ. Behav. Hum. Perform.
, 6
(6
), pp. 649
–744
.43.
Finucane
, M. L.
Alhakami
, A.
Slovic
, P.
Johnson
, S. M.
2000
, “The Affect Heuristic in Judgments of Risks and Benefits
,” J. Behav. Decis. Mak.
, 13
(1
), pp. 1
–17
.44.
Macy
, M. W.
Willer
, R.
2002
, “From Factors to Factors: Computational Sociology and Agent-Based Modeling
,” Annu. Rev. Sociol.
, 28
(1
), pp. 143
–166
.45.
Macal, C. M.
, and North, M. J
., 2008
, “Agent-Based Modeling and Simulation
,” Proceedings of the 40th Conference on Winter Simulation
, Miami, FL, Dec. 7–10, pp. 101
–112
.46.
McComb
, C.
Cagan
, J.
Kotovsky
, K.
2016
, “Drawing Inspiration From Human Design Teams For Better Search And Optimization: The Heterogeneous Simulated Annealing Teams Algorithm
,” ASME J. Mech. Des.
, 138
(4
), p. 044501
.47.
Triki
, E.
Collette
, Y.
Siarry
, P.
2005
, “A Theoretical Study on the Behavior of Simulated Annealing Leading to a New Cooling Schedule
,” Eur. J. Oper. Res.
, 166
(1
), pp. 77
–92
.48.
Patton
, M. Q.
2002
, Qualitative Research and Evaluation Methods
, SAGE
, Thousand Oaks, CA.49.
Anderson
, J. C.
Rungtusanatham
, M.
1994
, “A Theory of Quality Management Underlying the Deming Management Method
,” Acad. Manag. Rev.
, 19
(3
), pp. 472
–509
.50.
Azzalini
, A.
1985
, “A Class of Distributions Which Includes the Normal Ones
,” Scand. J. Stat.
, 12
(2
), pp. 171
–178
.https://www.jstor.org/stable/461598251.
Glaser
, B. G.
Strauss
, A. L.
1967
, The Discovery of Grounded Theory: Strategies for Qualitative Research
, Aldine Transactions, New Brunswick, NJ.52.
Corbin
, J. M.
Strauss
, A.
2008
, Basics of Qualitative Research: Techniques and Procedures for Developing Grounded Theory
, SAGE
, Thousand Oaks, CA.Copyright © 2018 by ASME
You do not currently have access to this content.
