The objective of this work is to explore the possible biases that individuals may have toward the perceived functionality of machine generated designs, compared to human created designs. Toward this end, 1187 participants were recruited via Amazon mechanical Turk (AMT) to analyze the perceived functional characteristics of both human created two-dimensional (2D) sketches and sketches generated by a deep learning generative model. In addition, a computer simulation was used to test the capability of the sketched ideas to perform their intended function and explore the validity of participants' responses. The results reveal that both participants and computer simulation evaluations were in agreement, indicating that sketches generated via the deep generative design model were more likely to perform their intended function, compared to human created sketches used to train the model. The results also reveal that participants were subject to biases while evaluating the sketches, and their age and domain knowledge were positively correlated with their perceived functionality of sketches. The results provide evidence that supports the capabilities of deep learning generative design tools to generate functional ideas and their potential to assist designers in creative tasks such as ideation.

References

References
1.
Chandrasegaran
,
S. K.
,
Ramani
,
K.
,
Sriram
,
R. D.
,
Horváth
,
I.
,
Bernard
,
A.
,
Harik
,
R. F.
, and
Gao
,
W.
,
2013
, “
The Evolution, Challenges, and Future of Knowledge Representation in Product Design Systems
,”
Comput. Des.
,
45
(
2
), pp.
204
228
.https://www.sciencedirect.com/science/article/pii/S0010448512001741
2.
Liapis
,
A.
,
Yannakakis
,
G. N.
,
Alexopoulos
,
C.
, and
Lopes
,
P.
,
2016
, “
Can Computers Foster Human User's Creativity? Theory and Practice of Mixed-Initiative Co-Creativity
,”
Digital Cult. Educ.
,
8
(
2
), pp.
136
153
.http://www.digitalcultureandeducation.com/uncategorized/liapis-html/
3.
Burnap
,
A.
,
Lui
,
Y.
,
Pan
,
Y.
,
Lee
,
H.
,
Gonzalez
,
R.
, and
Papalambors
,
P.
,
2016
, “
Estimating and Exploring the Product Form Design Space Using Deep Generative Models
,”
ASME
Paper No. DETC2016-60091.
4.
Dering
,
M. L.
, and
Tucker
,
C. S.
,
2017
, “
Generative Adversarial Networks for Increasing the Veracity of Big Data
,”
IEEE International Conference on Big Data
(
BIGDATA
), Boston, MA, Dec. 11–14, pp.
2513
2520
.
5.
Boden
,
M. A.
,
2004
,
The Creative Mind: Myths and Mechanisms
,
2nd ed.
,
Routledge
, New York.
6.
Kazi
,
R. H.
,
Grossman
,
T.
,
Cheong
,
H.
,
Hashemi
,
A.
, and
Fitzmaurice
,
G.
,
2017
, “
DreamSketch: Early Stage 3D Design Explorations With Sketching and Generative Design
,”
30th Annual ACM Symposium on User Interface Software and Technology
, Quebec City, QC, Canada, Oct. 22–25, pp.
401
414
.
7.
Lopez
,
C. E.
, and
Tucker
,
C. S.
,
2018
, “
Human Validation of Computer Versus Human Generated Design Sketches
,”
ASME
Paper No.
DETC2018-85698.
8.
Rietzschel
,
E. F.
,
Nijstad
,
B. A.
, and
Stroebe
,
W.
,
2006
, “
Productivity is Not Enough: A Comparison of Interactive and Nominal Brainstorming Groups on Idea Generation and Selection
,”
J. Exp. Soc. Psychol.
,
42
(
2
), pp.
244
251
.
9.
Toh
,
C. A.
,
Strohmetz
,
A. A.
, and
Miller
,
S. R.
,
2016
, “
The Effects of Gender and Idea Goodness on Ownership Bias in Engineering Design Education
,”
ASME J. Mech. Des.
,
138
(
10
), p.
101105
.
10.
Toh
,
C. A.
,
Patel
,
A. H.
,
Strohmetz
,
A. A.
, and
Miller
,
S. R.
,
2015
, “
My Idea is Best! Ownership Bias and Its Influence on Engineering Concept Selection
,”
ASME
Paper No. DETC2015-46478.
11.
Zheng
,
X.
, and
Miller
,
S. R.
,
2017
, “
Risky Business: The Driving Factors of Creative Risk Taking Attitudes in Engineering Design Industry
,”
ASME
Paper No. DETC2017-67799.
12.
Thomson
,
M. E.
,
Önkal
,
D.
,
Avcioǧlu
,
A.
, and
Goodwin
,
P.
,
2004
, “
Aviation Risk Perception: A Comparison Between Experts and Novices
,”
Risk Anal.
,
24
(
6
), pp.
1585
1595
.
13.
Arning
,
K.
, and
Ziefle
,
M.
,
2007
, “
Understanding Age Differences in PDA Acceptance and Performance
,”
Comput. Human Behav.
,
23
(
6
), pp.
2904
2927
.
14.
Wang
,
Y.-S.
,
Wu
,
M.-C.
, and
Wang
,
H.-Y.
,
2009
, “
Investigating the Determinants and Age and Gender Differences in the Acceptance of Mobile Learning
,”
Br. J. Educ. Technol.
,
40
(
1
), pp.
92
118
.
15.
Venkatesh
,
V.
,
Morris
,
M. G.
,
Davis
,
G. B.
, and
Davis
,
F. D.
,
2003
, “
User Acceptance of Information Technology: Toward a Unified View
,”
MIS Q.
,
27
(
3
), pp.
425
478
.
16.
Orsborn
,
S.
,
Cagan
,
J.
, and
Boatwright
,
P.
,
2009
, “
Quantifying Aesthetic Form Preference in a Utility Function
,”
ASME J. Mech. Des.
,
131
(
6
), p.
061001
.
17.
Reid
,
T. N.
,
Gonzalez
,
R. D.
, and
Papalambros
,
P. Y.
,
2010
, “
Quantification of Perceived Environmental Friendliness for Vehicle Silhouette Design
,”
ASME J. Mech. Des.
,
132
(
10
), p.
101010
.
18.
Schmidhuber
,
J.
,
2015
, “
Deep Learning in Neural Networks: An Overview
,”
Neural Networks
,
61
, pp.
85
117
.
19.
Ha
,
D.
, and
Eck
,
D.
,
2018
, “
A Neural Representation of Sketch Drawings
,” Sixth International Conference on Learning Representations, Vancouver, BC, Canada, pp. 1–16.
20.
Chen
,
Y.
,
Tu
,
S.
,
Yi
,
Y.
, and
Xu
,
L.
,
2017
, “
Sketch-pix2seq: A Model to Generate Sketches of Multiple Categories
,” e-print
arXiv:1709.04121
.https://arxiv.org/abs/1709.04121
21.
Achlioptas
,
P.
,
Diamanti
,
O.
,
Mitliagkas
,
I.
, and
Guibas
,
L.
,
2017
, “
Learning Representations and Generative Models for 3D Point Clouds
,” 35th International Conference on Machine Learning, Stockholm, Sweden, July 10–15, pp. 1–20.
22.
Bodén
,
M.
,
2001
, “
A Guide to Recurrent Neural Networks and Backpropagation
,” Halmstad, Sweden, Dallas Project SICS Technical Report T2002:03.
23.
Goodfellow
,
I.
,
Pouget-Abadie
,
J.
,
Mirza
,
M.
,
Xu
,
B.
,
Warde-Farley
,
D.
,
Ozair
,
S.
,
Courville
,
A.
, and
Bengio
,
Y.
,
2014
, “
Generative Adversarial Nets
,”
27th International Conference on Neural Information Processing Systems (NIPS)
, Montreal, QC, Canada, Dec. 8–13, pp.
2672
2680
.
24.
Dosovitskiy
,
A.
,
Springenberg
,
J. T.
,
Tatarchenko
,
M.
, and
Brox
,
T.
,
2017
, “
Learning to Generate Chairs, Tables and Cars With Convolutional Networks
,”
IEEE Trans. Pattern Anal. Mach. Intell.
,
39
(
4
), pp.
692
705
.
25.
Theis
,
L.
,
Oord
,
A. V. D.
, and
Bethge
,
M.
,
2016
, “
A Note on the Evaluation of Generative Models
,” International Conference of Learning Representations, San Juan, Puerto Rico, May 2–4, pp. 1–10.
26.
Poetz
,
M. K.
, and
Schreier
,
M.
,
2012
, “
The Value of Crowdsourcing: Can Users Really Compete With Professionals in Generating New Product Ideas?
,”
J. Prod. Innov. Manag.
,
29
(
2
), pp.
245
256
.
27.
Wang
,
G. G.
, and
Shan
,
S.
,
2007
, “
Review of Metamodeling Techniques in Support of Engineering Design Optimization
,”
ASME J. Mech. Des.
,
129
(
4
), pp.
370
380
.
28.
Dering
,
M.
, and
Tucker
,
C.
,
2017
, “
A Convolutional Neural Network Model for Predicting a Product's Function, Given Its Form
,”
ASME J. Mech. Des.
,
139
(
11
), pp.
1
14
.http://mechanicaldesign.asmedigitalcollection.asme.org/article.aspx?articleid=2645713
29.
Buxton
,
B.
,
2010
,
Sketching User Experiences: Getting the Design Right and the Right Design
,
Morgan Kaufmann
, San Francisco, CA.
30.
Goel
,
V.
,
1997
, “
Sketches of Thought
,”
Des. Stud.
,
18
(
1
), pp.
129
130
.https://www.sciencedirect.com/science/article/pii/S0142694X97832887
31.
Rodgers
,
P. A.
,
Green
,
G.
, and
McGown
,
A.
,
2000
, “
Using Concept Sketches to Track Design Progress
,”
Des. Stud.
,
21
(
5
), pp.
451
464
.
32.
Van Der Lugt
,
R.
,
2005
, “
How Sketching Can Affect the Idea Generation Process in Design Group Meetings
,”
Des. Stud.
,
26
(
2
), pp.
101
112
.
33.
Yang
,
M. C.
,
2009
, “
Observations on Concept Generation and Sketching in Engineering Design
,”
Res. Eng. Des.
,
20
(
1
), pp.
1
11
.
34.
Macomber
,
B.
, and
Yang
,
M. C.
,
2011
, “
The Role of Sketch Finish and Style in User Responses to Early Stage Design Concepts
,”
ASME
Paper No. DETC2011-48714.
35.
Häggman
,
A.
,
Tsai
,
G.
,
Elsen
,
C.
,
Honda
,
T.
, and
Yang
,
M. C.
,
2015
, “
Connections Between the Design Tool, Design Attributes, and User Preferences in Early Stage Design
,”
ASME J. Mech. Des.
,
137
(
7
), p.
071101
.
36.
Cunningham
,
J.
, and
Tucker
,
C. S.
,
2018
, “
A Validation Neural Network (VNN) Metamodel for Predicting the Performance of Deep Generative Designs
,”
ASME Paper No. DETC2018-86299
.
37.
Ren
,
Y.
,
Burnap
,
A.
, and
Papalambros
,
P.
,
2013
, “
Quantification of Perceptual Design Attributes Using a Crowd
,”
19th International Conference on Engineering Design
, Seoul, South Korea, pp.
19
22
.
38.
Toh
,
C. A.
,
Miele
,
L. M.
, and
Miller
,
S. R.
,
2016
, “
Which One Should I Pick? Concept Selection in Engineering Design Industry
,”
ASME
Paper No. DETC2015-46522.
39.
Cox
,
D.
, and
Cox
,
A. D.
,
2002
, “
Beyond First Impressions: The Effects of Repeated Exposure on Consumer Liking of Visually Complex and Simple Product Designs
,”
J. Acad. Mark. Sci.
,
30
(
2
), pp.
119
130
.
40.
Mueller
,
J. S.
,
Melwani
,
S.
, and
Goncalo
,
J. A.
,
2012
, “
The Bias Against Creativity: Why People Desire but Reject Creative Ideas
,”
Psychol. Sci.
,
23
(
1
), pp.
13
17
.
41.
Toh
,
C. A.
, and
Miller
,
S. R.
,
2014
, “
The Role of Individual Risk Attitudes on the Selection of Creative Concepts in Engineering Design
,”
ASME
Paper No. DETC2014-35106.
42.
Burnap
,
A.
,
Gerth
,
R.
,
Gonzalez
,
R.
, and
Papalambros
,
P. Y.
,
2017
, “
Identifying Experts in the Crowd for Evaluation of Engineering Designs
,”
J. Eng. Des.
,
28
(
5
), pp.
317
337
.
43.
W.
,
I. J.
,
Nap
,
H. H.
,
De Kort
,
Y.
, and
Poels
,
K.
,
2007
, “
Digital Game Design for Elderly Users
,”
Conference on Future Play
, Toronto, ON, Canada, Nov. 14–17, pp.
17
22
.
44.
Parasuraman
,
R.
, and
Manzey
,
D. H.
,
2010
, “
Complacency and Bias in Human Use of Automation: An Attentional Integration
,”
Hum. Factors
,
52
(
3
), pp.
381
410
.
45.
Mosier
,
K. L.
, and
Skitka
,
L. J.
,
1996
, “
Human Decision Makers and Automated Decision Aids: Made for Each Other
?,”
Automation and Human Performance
,
Erlbaum
, Hillsdale, NJ.
46.
Lee
,
J. D.
, and
See
,
K. A.
,
2004
, “
Trust in Automation: Designing for Appropriate Reliance
,”
Hum. Factors
,
46
(
1
), pp.
50
80
.http://journals.sagepub.com/doi/10.1518/hfes.46.1.50_30392
47.
Dzindolet
,
M. T.
,
Pierce
,
L. G.
,
Beck
,
H. P.
, and
Dawe
,
L. A.
,
2002
, “
The Perceived Utility of Human and Automated Aids in a Visual Detection Task
,”
Hum. Factors
,
44
(
1
), pp.
79
94
.
48.
Le
,
Q.
, and
Panchal
,
J. H.
,
2011
, “
Modeling the Effect of Product Architecture on Mass-Collaborative Processes
,”
ASME J. Comput. Inf. Sci. Eng.
,
11
(
1
), p.
011003
.http://computingengineering.asmedigitalcollection.asme.org/article.aspx?articleid=1402261
49.
Jongejan
,
J.
,
Rowley
,
H.
,
Kawashima
,
T.
,
Kim
,
J.
, and
Fox-Gieg
,
N.
,
2016
, “
The Quick, Draw!—A.I. Experiment
,” Mount View, CA, accessed Feb. 17, 2018, https://quickdraw.withgoogle.com/
50.
Buchanan
,
T.
,
2000
,
Psychological Experiments on the Internet
,
Academic Press
, Oxford, UK.
51.
Mason
,
W.
, and
Suri
,
S.
,
2012
, “
Conducting Behavioral Research on Amazon's Mechanical Turk
,”
Behav. Res. Methods
,
44
(
1
), pp.
1
23
.https://link.springer.com/article/10.3758/s13428-011-0124-6
52.
Unity
,
2017
, “
Unity—Game Engine
,” San Francisco, CA, accessed May 31, 2018, https://www.unity3d.com
53.
González
,
J. D.
,
Escobar
,
J. H.
,
Sánchez
,
H.
,
De La Hoz
,
J.
, and
Beltrán
,
J. R.
,
2017
, “
2D and 3D Virtual Interactive Laboratories of Physics on Unity Platform
,”
J. Phys.: Conf. Ser.
,
935
(
1
), p.
012069
.http://iopscience.iop.org/article/10.1088/1742-6596/935/1/012069
54.
Ballu
,
A.
,
Yan
,
X.
,
Blanchard
,
A.
,
Clet
,
T.
,
Mouton
,
S.
, and
Niandou
,
H.
,
2016
, “
Virtual Metrology Laboratory for e-Learning
,”
Procedia CIRP
,
43
(1), pp.
148
153
.https://www.sciencedirect.com/science/article/pii/S2212827116003929
55.
Cortina
,
J. M.
,
1993
, “
What is Coefficient Alpha? an Examination of Theory and Applications
,”
J. Appl. Psychol.
,
78
(
1
), pp.
98
104
.http://psycnet.apa.org/record/1993-19965-001
56.
Cohen
,
J.
,
1988
,
Statistical Power Analysis for the Behavioral Sciences
,
2nd ed.
,
Routledge
, New York.
You do not currently have access to this content.