Abstract

Engineering design involves information acquisition decisions such as selecting designs in the design space for testing, selecting information sources, and deciding when to stop design exploration. Existing literature has established normative models for these decisions, but there is lack of knowledge about how human designers make these decisions and which strategies they use. This knowledge is important for accurately modeling design decisions, identifying sources of inefficiencies, and improving the design process. Therefore, the primary objective in this study is to identify models that provide the best description of a designer’s information acquisition decisions when multiple information sources are present and the total budget is limited. We conduct a controlled human subject experiment with two independent variables: the amount of fixed budget and the monetary incentive proportional to the saved budget. By using the experimental observations, we perform Bayesian model comparison on various simple heuristic models and expected utility (EU)-based models. As expected, the subjects’ decisions are better represented by the heuristic models than the EU-based models. While the EU-based models result in better net payoff, the heuristic models used by the subjects generate better design performance. The net payoff using heuristic models is closer to the EU-based models in experimental treatments where the budget is low and there is incentive for saving the budget. This indicates the potential for nudging designers’ decisions toward maximizing the net payoff by setting the fixed budget at low values and providing monetary incentives proportional to saved budget.

References

References
1.
Hazelrigg
,
G. A.
,
1998
, “
A Framework for Decision-Based Engineering Design
,”
ASME J. Mech. Des.
,
120
(
4
), pp.
653
658
. 10.1115/1.2829328
2.
Chaudhari
,
A. M.
,
Sha
,
Z.
, and
Panchal
,
J. H.
,
2018
, “
Analyzing Participant Behaviors in Design Crowdsourcing Contests Using Causal Inference on Field Data
,”
ASME J. Mech. Des.
,
140
(
9
), p.
091401
. 10.1115/1.4040166
3.
Safarkhani
,
S.
,
Bilionis
,
I.
, and
Panchal
,
J. H.
,
2018
, “
Understanding the Effect of Task Complexity and Problem-Solving Skills on the Design Performance of Agents in Systems Engineering
,”
44th Design Automation Conference
,
Quebec City, Canada
,
Aug. 26–29
, Vol.
2A
, p.
V02AT03A060
.
4.
Hazelrigg
,
G. A.
,
1999
, “
An Axiomatic Framework for Engineering Design
,”
ASME J. Mech. Des.
,
121
(
3
), pp.
342
347
. 10.1115/1.2829466
5.
Marston
,
M.
,
Allen
,
J. K.
, and
Mistree
,
F.
,
2000
, “
The Decision Support Problem Technique: Integrating Descriptive and Normative Approaches in Decision Based Design
,”
Eng. Valuation Cost Anal.
,
3
(
2
), pp.
107
129
.
6.
Wassenaar
,
H. J.
, and
Chen
,
W.
,
2003
, “
An Approach to Decision-Based Design With Discrete Choice Analysis for Demand Modeling
,”
ASME J. Mech. Des.
,
125
(
3
), pp.
490
497
. 10.1115/1.1587156
7.
Moore
,
R. A.
,
Romero
,
D. A.
, and
Paredis
,
C. J.
,
2014
, “
Value-Based Global Optimization
,”
ASME J. Mech. Des.
,
136
(
4
), p.
041003
. 10.1115/1.4026281
8.
Fishburn
,
P. C.
,
1988
, “
Expected Utility: An Anniversary and a New Era
,”
J. Risk Uncertainty
,
1
(
3
), pp.
267
283
. 10.1007/BF00056138
9.
Kahneman
,
D.
, and
Tversky
,
A.
,
1979
, “
Prospect Theory: An Analysis of Decision Under Risk
,”
Econometrica
,
47
(
2
), pp.
263
292
. 10.2307/1914185
10.
Tebay
,
R.
,
Atherton
,
J.
, and
Wearne
,
S.
,
1984
, “
Mechanical Engineering Design Decisions: Instances of Practice Compared With Theory
,”
Proc. Inst. Mech. Eng., Part B Manage. Eng. Manuf.
,
198
(
2
), pp.
87
96
.
11.
Finger
,
S.
, and
Dixon
,
J. R.
,
1989
, “
A Review of Research in Mechanical Engineering Design. Part I: Descriptive, Prescriptive, and Computer-Based Models of Design Processes
,”
Res. Eng. Des.
,
1
(
1
), pp.
51
67
. 10.1007/BF01580003
12.
Tversky
,
A.
, and
Kahneman
,
D.
,
1992
, “
Advances in Prospect Theory: Cumulative Representation of Uncertainty
,”
J. Risk Uncertainty
,
5
(
4
), pp.
297
323
. 10.1007/BF00122574
13.
Gigerenzer
,
G.
, and
Gaissmaier
,
W.
,
2011
, “
Heuristic Decision Making
,”
Annu. Rev. Psychol.
,
62
, pp.
451
482
. 10.1146/annurev-psych-120709-145346
14.
Simon
,
H. A.
,
1955
, “
A Behavioral Model of Rational Choice
,”
Q. J. Econ.
,
69
(
1
), p.
99
. 10.2307/1884852
15.
Gonzalez
,
C.
,
Lerch
,
J. F.
, and
Lebiere
,
C.
,
2003
, “
Instance-Based Learning in Dynamic Decision Making
,”
Cognit. Sci.
,
27
(
4
), pp.
591
635
. 10.1207/s15516709cog2704_2
16.
Panchal
,
J. H.
, and
Szajnfarber
,
Z.
,
2017
, “
Experiments in Systems Engineering and Design Research
,”
Syst. Eng.
,
20
(
6
), pp.
529
541
. 10.1002/sys.21415
17.
Falk
,
A.
, and
Heckman
,
J. J.
,
2009
, “
Lab Experiments Are a Major Source of Knowledge in the Social Sciences
,”
Science
,
326
(
5952
), pp.
535
538
. 10.1126/science.1168244
18.
Simon
,
H. A.
, and
Newell
,
A.
,
1971
, “
Human Problem Solving: The State of the Theory in 1970
,”
Am. Psychologist
,
26
(
2
), p.
145
. 10.1037/h0030806
19.
Shergadwala
,
M.
,
Bilionis
,
I.
,
Kannan
,
K.
, and
Panchal
,
J. H.
,
2018
, “
Quantifying the Impact of Domain Knowledge and Problem Framing on Sequential Decisions in Engineering Design
,”
ASME J. Mech. Des.
,
140
(
10
), p.
101402
. 10.1115/1.4040548
20.
Freriks
,
H.
,
Heemels
,
W.
,
Muller
,
G.
, and
Sandee
,
J.
,
2006
, “
On the Systematic Use of Budget-Based Design
,”
INCOSE International Symposium
,
Orlando, FL
,
July 8–14
, Vol.
16
,
Wiley Online Library
, pp.
788
803
.
21.
Moroşan
,
P.-D.
,
Bourdais
,
R.
,
Dumur
,
D.
, and
Buisson
,
J.
,
2010
, “
Building Temperature Regulation Using a Distributed Model Predictive Control
,”
Energy Build.
,
42
(
9
), pp.
1445
1452
. 10.1016/j.enbuild.2010.03.014
22.
Hor
,
Y. S.
,
Williams
,
A. J.
,
Checkelsky
,
J. G.
,
Roushan
,
P.
,
Seo
,
J.
,
Xu
,
Q.
,
Zandbergen
,
H. W.
,
Yazdani
,
A.
,
Ong
,
N.
, and
Cava
,
R. J.
,
2010
, “
Superconductivity in CuxBi2Se3 and Its Implications for Pairing in the Undoped Topological Insulator
,”
Phys. Rev. Lett.
,
104
(
5
), p.
057001
. 10.1103/PhysRevLett.104.057001
23.
Nikolaidis
,
E.
,
Ghiocel
,
D. M.
, and
Singhal
,
S.
,
2004
,
Engineering Design Reliability Handbook
,
CRC Press
,
Boca Raton, FL
.
24.
Loch
,
C. H.
,
Terwiesch
,
C.
, and
Thomke
,
S.
,
2001
, “
Parallel and Sequential Testing of Design Alternatives
,”
Manage. Sci.
,
47
(
5
), pp.
663
678
. 10.1287/mnsc.47.5.663.10480
25.
Dahan
,
E.
, and
Mendelson
,
H.
,
1997
, “
Optimal Sequential and Parallel Prototyping Policies
,”
Innovation in Technology Management, The Key to Global Leadership, PICMET
,
Portland, OR
,
July 31
,
IEEE
.
26.
Jones
,
D. R.
,
2001
, “
A Taxonomy of Global Optimization Methods Based on Response Surfaces
,”
J. Global Optim.
,
21
(
4
), pp.
345
383
. 10.1023/A:1012771025575
27.
Jones
,
D. R.
,
Schonlau
,
M.
, and
Welch
,
W. J.
,
1998
, “
Efficient Global Optimization of Expensive Black-Box Functions
,”
J. Global Optim.
,
13
(
4
), pp.
455
492
. 10.1023/A:1008306431147
28.
Gramacy
,
R. B.
, and
Lee
,
H. K. H.
,
2011
, “Optimization Under Unknown Constraints,”
Bayesian Statistics 9
,
J.
Bernardo
,
M.
Bayarri
,
J.
Berger
,
A.
Dawid
,
D.
Heckerman
,
A.
Smith
, and
M.
West
, eds.,
Oxford University Press
,
Oxford
, pp.
229
256
.
29.
Auer
,
P.
,
2002
, “
Using Confidence Bounds for Exploitation-Exploration Trade-Offs
,”
J. Mach. Learn. Res.
,
3
, pp.
397
422
.
30.
Chaudhari
,
A. M.
, and
Panchal
,
J. H.
,
2019
, “
An Experimental Study of Human Decisions in Sequential Information Acquisition in Design: Impact of Cost and Task Complexity
,”
Research Into Design for a Connected World
,
Bangalore, India
, Vol.
134
, pp.
321
332
.
31.
Chaudhari
,
A. M.
,
Bilionis
,
I.
, and
Panchal
,
J. H.
,
2018
, “
How Do Designers Choose Among Multiple Noisy Information Sources in Engineering Design Optimization? An Experimental Study
,”
44th Design Automation Conference
,
Quebec City, Canada
,
Aug. 26–29
, Vol.
2A
, p.
V02AT03A021
.
32.
Panchal
,
J. H.
,
Sha
,
Z.
, and
Kannan
,
K. N.
,
2017
, “
Understanding Design Decisions Under Competition Using Games With Information Acquisition and a Behavioral Experiment
,”
ASME J. Mech. Des.
,
139
(
9
), p.
091402
. 10.1115/1.4037253
33.
Shafir
,
E.
, and
Tversky
,
A.
,
1992
, “
Thinking Through Uncertainty: Nonconsequential Reasoning and Choice
,”
Cognit. Psychol.
,
24
(
4
), pp.
449
474
. 10.1016/0010-0285(92)90015-T
34.
Chen
,
D. L.
,
Schonger
,
M.
, and
Wickens
,
C.
,
2016
, “
oTree–An Open-Source Platform for Laboratory, Online, and Field Experiments
,”
J. Behav. Exp. Finance
,
9
, pp.
88
97
. 10.1016/j.jbef.2015.12.001
35.
Strack
,
F.
,
1992
, “Order Effects in Survey Research: Activation and Information Functions of Preceding Questions,”
Context Effects in Social and Psychological Research
,
N.
Schwarz
, and
S.
Sudman
, eds.,
Springer
,
New York
, pp.
23
34
.
36.
Kingma
,
D. P.
, and
Welling
,
M.
,
2013
, “
Auto-Encoding Variational Bayes
,” .
37.
Gelman
,
A.
,
Hwang
,
J.
, and
Vehtari
,
A.
,
2014
, “
Understanding Predictive Information Criteria for Bayesian Models
,”
Stat. Comput.
,
24
(
6
), pp.
997
1016
. 10.1007/s11222-013-9416-2
38.
Kucukelbir
,
A.
,
Tran
,
D.
,
Ranganath
,
R.
,
Gelman
,
A.
, and
Blei
,
D. M.
,
2017
, “
Automatic Differentiation Variational Inference
,”
J. Mach. Learn. Res.
,
18
(
1
), pp.
430
474
.
39.
Salvatier
,
J.
,
Wiecki
,
T. V.
, and
Fonnesbeck
,
C.
,
2016
, “
Probabilistic Programming in Python Using PyMC3
,”
PeerJ Comput. Sci.
,
2
, p.
e55
. 10.7717/peerj-cs.55
40.
Sha
,
Z.
,
Chaudhari
,
A. M.
, and
Panchal
,
J. H.
,
2019
, “
Modeling Participation Behaviors in Design Crowdsourcing Using a Bipartite Network-Based Approach
,”
J. Comput. Inf. Sci. Eng.
,
19
(
3
), p.
031010
. 10.1115/1.4042639
41.
Borji
,
A.
, and
Itti
,
L.
,
2013
, “
Bayesian Optimization Explains Human Active Search
,”
Advances in Neural Information Processing Systems
,
Lake Tahoe, NV
,
Dec. 5–10
, pp.
55
63
.
42.
Griffiths
,
T. L.
,
Lucas
,
C.
,
Williams
,
J.
, and
Kalish
,
M. L.
,
2009
, “Modeling Human Function Learning With Gaussian Processes,”
Advances in Neural Information Processing Systems
,
Vancouver
, BC, pp.
553
560
.
43.
Lucas
,
C. G.
,
Griffiths
,
T. L.
,
Williams
,
J. J.
, and
Kalish
,
M. L.
,
2015
, “
A Rational Model of Function Learning
,”
Psychon. Bull. Rev.
,
22
(
5
), pp.
1193
1215
. 10.3758/s13423-015-0808-5
44.
Binder
,
W. R.
, and
Paredis
,
C. J.
,
2017
, “
Optimization Under Uncertainty Versus Algebraic Heuristics: A Research Method for Comparing Computational Design Methods
,”
43rd Design Automation Conference
,
Cleveland, OH
,
Aug. 6–9
, Vol.
2B
, p.
V02BT03A057
.
45.
Sissoko
,
T. M.
,
Jankovic
,
M.
,
Paredis
,
C. J. J.
, and
Landel
,
E.
,
2018
, “
An Empirical Study of a Decision-Making Process Supported by Simulation in the Automotive Industry
,”
30th International Conference on Design Theory and Methodology
,
Quebec City, Canada
,
Aug. 26–29
.
46.
Tiong
,
E.
,
Seow
,
O.
,
Camburn
,
B.
,
Teo
,
K.
,
Silva
,
A.
,
Wood
,
K. L.
,
Jensen
,
D. D.
, and
Yang
,
M. C.
,
2018
, “
The Economies and Dimensionality of Design Prototyping: Value, Time, Cost and Fidelity
,”
ASME J. Mech. Des.
,
143
(
3
), p.
031105
. 10.1115/DETC2018-85747
You do not currently have access to this content.