Abstract

We developed a new method to link designer emotions with corresponding designer activities while using computer-aided design (cad) software. Our method employs automated facial emotion detection software and cursor tracking. We applied this method via an experiment with nine participants, each working with the same synchronously collaborative cad platform, and assigned a series of cad tasks in one of two distinct working styles: single participants working by themselves and paired participants working together. We analyzed and compared trends in emotion for these two working styles. Pairs, on average per person, experienced higher levels of emotion (measured as joy, sadness, anger, contempt, fear, and surprise) than individuals. We linked occurrences of each emotional response to their antecedent activities in the cad environment (navigating the model tree, sketching in the graphics area, making selections in the feature menu, and communicating using the chat window). Using a logistic regression analysis, we revealed statistically significant trends linking emotions and cad events, and we found that some emotions are more likely to occur with certain designer actions in the cad software. The method and conclusions presented in this paper allow us to better understand designer emotions in traditional and collaborative cad, which link to the established relationships between emotion and designer satisfaction, creativity, performance, and other outcomes increasingly valued by engineering designers and managers in virtually collaborative environments.

References

References
1.
Demirbilek
,
O.
, and
Sener
,
B.
,
2003
, “
Product Design, Semantics and Emotional Response
,”
Ergonomics
,
46
(
13/14
), pp.
1346
1360
. 10.1080/00140130310001610874
2.
Damen
,
N.
, and
Toh
,
C.
,
2019
, “
Designing for Trust: Understanding the Role of Agent Gender and Location on User Perceptions of Trust in Home Automation
,”
ASME J. Mech. Des.
,
141
(
6
), p.
061106
. 10.1115/1.4042223
3.
Bao
,
Q.
,
Burnell
,
E.
,
Hughes
,
A. M.
, and
Yang
,
M. C.
,
2019
, “
Investigating User Emotional Responses to Eco-Feedback Designs
,”
ASME J. Mech. Des.
,
141
(
2
), p.
021103
. 10.1115/1.4042007
4.
Sas
,
C.
, and
Zhang
,
C.
,
2010
, “
Do Emotions Matter in Creative Design?
DIS ‘10: Proceedings of the 8th ACM Conference on Designing Interactive Systems
,
Aarhus, Denmark
, pp.
372
375
.
5.
National Academies of Sciences, Engineering, and Medicine
,
2017
,
Information Technology and the U.S. Workforce: Where Are We and Where Do We Go from Here?
The National Academies Press
,
Washington, DC
.
6.
Bailey
,
D. E.
,
Leonardi
,
P. M.
, and
Barley
,
S. R.
,
2012
, “
The Lure of the Virtual
,”
Organ. Sci.
,
23
(
5
), pp.
1485
1504
. 10.1287/orsc.1110.0703
7.
Balijepally
,
V.
,
Mahapatra
,
R.
,
Nerur
,
S.
, and
Price
,
K. H.
,
2009
, “
Are Two Heads Better Than One for Software Development? The Productivity Paradox of Pair Programming
,”
MIS Q.
,
33
(
1
), pp.
91
118
. 10.2307/20650280
8.
Baas
,
M.
,
De Dreu
,
C. K. W.
, and
Nijstad
,
B. A.
,
2008
, “
A Meta-Analysis of 25 Years of Mood-Creativity Research: Hedonic Tone, Activation, or Regulatory Focus?
,”
Psychol. Bull.
,
134
(
6
), pp.
779
806
. 10.1037/a0012815
9.
Barrett
,
L. F.
,
Lewis
,
M.
, and
Haviland-Jones
,
J. M.
,
2016
,
Handbook of Emotions
, 4th ed.,
Guilford Publications
,
New York
.
10.
Calvo
,
R. A.
, and
D’Mello
,
S.
,
2010
, “
Affect Detection: An Interdisciplinary Review of Models, Methods, and Their Applications
,”
IEEE Trans. Affect. Comput.
,
1
(
1
), pp.
18
37
. 10.1109/T-AFFC.2010.1
11.
Ekman
,
P.
,
1992
, “
Are There Basic Emotions?
,”
Psychol. Rev.
,
99
(
3
), pp.
550
553
. 10.1037/0033-295X.99.3.550
12.
Lindquist
,
K. A.
,
Wager
,
T. D.
,
Kober
,
H.
,
Bliss-Moreau
,
E.
, and
Barrett
,
L. F.
,
2012
, “
The Brain Basis of Emotion: A Meta-Analytic Review
,”
Behav. Brain Sci.
,
35
(
03
), pp.
121
143
. 10.1017/S0140525X11000446
13.
Ekman
,
P.
, and
Heider
,
K. G.
,
1988
, “
The Universality of a Contempt Expression: A Replication
,”
Motiv. Emot.
,
12
(
3
), pp.
303
308
. 10.1007/BF00993116
14.
Barrett
,
L. F.
,
Adolphs
,
R.
,
Marsella
,
S.
,
Martinez
,
A. M.
, and
Pollak
,
S. D.
,
2019
, “
Emotional Expressions Reconsidered: Challenges to Inferring Emotion From Human Facial Movements
,”
Psychol. Sci. Public Interest
,
20
(
1
), pp.
1
68
. 10.1177/1529100619832930
15.
Côté
,
S.
,
1999
, “
Affect and Performance in Organizational Settings
,”
Curr. Dir. Psychol. Sci.
,
8
(
2
), pp.
65
68
. 10.1111/1467-8721.00016
16.
Barsade
,
S. G.
, and
Gibson
,
D. E.
,
2007
, “
Why Does Affect Matter in Organizations?
,”
Acad. Manage. Perspect.
,
21
(
1
), pp.
36
59
. 10.5465/amp.2007.24286163
17.
Kaplan
,
S.
,
Bradley
,
J. C.
,
Luchman
,
J. N.
, and
Haynes
,
D.
,
2009
, “
On the Role of Positive and Negative Affectivity in Job Performance: A Meta-Analytic Investigation
,”
J. Appl. Psychol.
,
94
(
1
), pp.
162
176
. 10.1037/a0013115
18.
Amabile
,
T. M.
,
Barsade
,
S. G.
,
Mueller
,
J. S.
, and
Staw
,
B. M.
,
2005
, “
Affect and Creativity at Work
,”
Adm. Sci. Q.
,
50
(
3
), pp.
367
403
. 10.2189/asqu.2005.50.3.367
19.
Niklas
,
C. D.
, and
Dormann
,
C.
,
2005
, “
The Impact of State Affect on Job Satisfaction
,”
Eur. J. Work Organ. Psychol.
,
14
(
4
), pp.
367
388
. 10.1080/13594320500348880
20.
Koriat
,
A.
,
2012
, “
When Are Two Heads Better Than One and Why?
Science
,
336
(
6079
), pp.
360
363
. 10.1126/science.1216549
21.
Allen
,
N. J.
, and
Hecht
,
T. D.
,
2004
, “
The ‘Romance of Teams’: Toward an Understanding of Its Psychological Underpinnings and Implications
,”
J. Occup. Organ. Psychol.
,
77
(
4
), pp.
439
461
. 10.1348/0963179042596469
22.
Behoora
,
I.
, and
Tucker
,
C. S.
,
2015
, “
Machine Learning Classification of Design Team Members’ Body Language Patterns for Real Time Emotional State Detection
,”
Des. Stud.
,
39
, pp.
100
127
. 10.1016/j.destud.2015.04.003
23.
Bezawada
,
S.
,
Hu
,
Q.
,
Gray
,
A.
,
Brick
,
T.
, and
Tucker
,
C.
,
2017
, “
Automatic Facial Feature Extraction for Predicting Designers’ Comfort With Engineering Equipment During Prototype Creation
,”
ASME J. Mech. Des.
,
139
(
2
), p.
021102
. 10.1115/1.4035428
24.
Hu
,
W.-L.
,
Booth
,
J. W.
, and
Reid
,
T.
,
2017
, “
The Relationship Between Design Outcomes and Mental States During Ideation
,”
ASME J. Mech. Des.
,
139
(
5
), p.
051101
. 10.1115/1.4036131
25.
Barsade
,
S. G.
, and
Knight
,
A. J.
,
2012
, “
Group Affect
,”
Curr. Dir. Psychol. Sci.
,
21
(
2
), pp.
119
123
. 10.1177/0963721412438352
26.
Liu
,
Y.
,
Ritchie
,
J. M.
,
Lim
,
T.
,
Kosmadoudi
,
Z.
,
Sivanathan
,
A.
, and
Sung
,
R. C. W.
,
2014
, “
A Fuzzy Psycho-Physiological Approach to Enable the Understanding of an Engineer’s Affect Status During CAD Activities
,”
CAD Comput. Aided Des.
,
54
, pp.
19
38
. 10.1016/j.cad.2013.10.007
27.
Sivanathan
,
A.
,
Lim
,
T.
,
Ritchie
,
J.
,
Sung
,
R.
,
Kosmadoudi
,
Z.
, and
Liu
,
Y.
,
2013
, “
The Application of Ubiquitous Multimodal Synchronous Data Capture in CAD
,”
Comput. Des.
,
59
, pp.
176
191
. 10.1016/j.cad.2013.10.001
28.
Ekman
,
P.
, and
Friesen
,
W.
,
1978
, “
Facial Signs of Emotional Experience
,”
J. Pers. Soc. Psychol.
,
39
(
6
), pp.
1123
1134
.
29.
Kleck
,
R. E.
,
Vaughan
,
R. C.
,
Cartwright-Smith
,
J.
,
Vaughan
,
K. B.
,
Colby
,
C. Z.
, and
Lanzetta
,
J. T.
,
1976
, “
Effects of Being Observed on Expressive, Subjective, and Physiological Responses to Painful Stimuli
,”
J. Pers. Soc. Psychol.
,
34
(
6
), pp.
1211
1218
. 10.1037/0022-3514.34.6.1211
30.
Tassinary
,
L.
,
Cacioppo
,
J.
, and
Geen
,
T.
,
1989
, “
A Psychometric Study of Surface Electrode Placements for Facial Electromyographic Recording: I. The Brow and Cheek Muscle Regions
,”
Psychophysiology
,
26
(
1
), pp.
1
16
. 10.1111/j.1469-8986.1989.tb03125.x
31.
Sayette
,
M. A.
,
Cohn
,
J. F.
,
Wertz
,
J. M.
,
Perrott
,
M. A.
, and
Parrott
,
D. J.
,
2001
, “
A Psychometric Evaluation of the Facial Action Coding System for Assessing Spontaneous Expression
,”
J. Nonverbal Behav.
,
25
(
3
), pp.
167
185
. 10.1023/A:1010671109788
32.
Ekman
,
P.
, and
Rosenberg
,
E.
,
1997
,
What the Face Reveals: Basic and Applied Studies of Spontaneous Expression Using the Facial Action Coding System
,
Oxford University Press
,
New York
.
33.
Senechal
,
T.
,
McDuff
,
D.
, and
El Kaliouby
,
R.
,
2015
, “
Facial Action Unit Detection Using Active Learning and an Efficient Non-Linear Kernel Approximation
,”
Proceedings of the IEEE International Conference on Computer Vision Workshop
,
Santiago, Chile
, pp.
10
18
.
34.
Anil
,
J.
, and
Suresh
,
L. P.
,
2016
, “
Literature Survey on Face and Face Expression Recognition
,”
Proceedings of the IEEE International Conference on Circuit, Power and Computing Technologies (ICCPCT)
,
Kumaracoil, India
, pp.
1
6
.
35.
McDuff
,
D.
,
El Kaliouby
,
R.
, and
Picard
,
R. W.
,
2012
, “
Crowdsourcing Facial Responses to Online Videos
,”
IEEE Trans. Affect. Comput.
,
3
(
4
), pp.
456
468
. 10.1109/T-AFFC.2012.19
36.
Chen
,
X.
,
Gao
,
S.
,
Yang
,
Y.
, and
Zhang
,
S.
,
2012
, “
Multi-Level Assembly Model for Top-Down Design of Mechanical Products
,”
CAD Comput. Aided Des.
,
44
(
10
), pp.
1033
1048
. 10.1016/j.cad.2010.12.008
37.
Horváth
,
I.
, and
Vroom
,
R. W.
,
2015
, “
Ubiquitous Computer Aided Design: A Broken Promise or a Sleeping Beauty?
,”
CAD Comput. Aided Des.
,
59
, pp.
161
175
. 10.1016/j.cad.2014.10.006
38.
Wu
,
D.
,
Terpenny
,
J.
, and
Schaefer
,
D.
,
2017
, “
Digital Design and Manufacturing on the Cloud: A Review of Software and Services
,”
Artif. Intell. Eng. Des. Anal. Manuf. AIEDAM
,
31
(
1
), pp.
104
118
. 10.1017/S0890060416000305
39.
Ostergaard
,
K. J.
, and
Summers
,
J. D.
,
2009
, “
Development of a Systematic Classification and Taxonomy of Collaborative Design Activities
,”
J. Eng. Des.
,
20
(
1
), pp.
57
81
. 10.1080/09544820701499654
40.
Maynard
,
M. T.
, and
Gilson
,
L. L.
,
2014
, “
The Role of Shared Mental Model Development in Understanding Virtual Team Effectiveness
,”
Group. Organ. Manage.
,
39
(
1
), pp.
3
32
. 10.1177/1059601113475361
41.
McComb
,
C.
,
Cagan
,
J.
, and
Kotovsky
,
K.
,
2017
, “
Optimizing Design Teams Based on Problem Properties: Computational Team Simulations and an Applied Empirical Test
,”
ASME J. Mech. Des.
,
139
(
4
), p.
041101
. 10.1115/1.4035793
42.
Red
,
E.
,
Holyoak
,
V.
,
Jensen
,
C. G.
,
Marshall
,
F.
,
Ryskamp
,
J.
, and
Xu
,
Y.
,
2010
, “
v-CAx: A Research Agenda for Collaborative Computer-Aided Applications
,”
Comput. Aided Des. Appl.
,
7
(
3
), pp.
1
15
. 10.3722/cadaps.2010.387-404
43.
Red
,
E.
,
Marshall
,
F.
,
Weerakoon
,
P.
, and
Jensen
,
C. G.
,
2013
, “
Considerations for Multi-User Decomposition of Design Spaces
,”
Comput. Aided Des. Appl.
,
10
(
5
), pp.
803
815
. 10.3722/cadaps.2013.803-815
44.
Briggs
,
J. C.
,
Hepworth
,
A. I.
,
Stone
,
B. R.
,
Coburn
,
J. Q.
,
Jensen
,
C. G.
, and
Red
,
E.
,
2015
, “
Integrated, Synchronous Multi-User Design and Analysis
,”
ASME J. Comput. Inf. Sci. Eng.
,
15
(
3
), p.
031002
. 10.1115/1.4029801
45.
Hepworth
,
A. I.
,
Tew
,
K.
,
Nysetvold
,
T.
,
Bennett
,
M.
, and
Greg Jensen
,
C.
,
2014
, “
Automated Conflict Avoidance in Multi-User CAD
,”
Comput. Aided Des. Appl.
,
11
(
2
), pp.
141
152
. 10.1080/16864360.2014.846070
46.
Eves
,
K.
,
Salmon
,
J.
,
Olsen
,
J.
, and
Fagergren
,
F.
,
2018
, “
A Comparative Analysis of Computer-Aided Design Team Performance With Collaboration Software
,”
Comput. Aided Des. Appl.
,
15
(
4
), pp.
476
487
. 10.1080/16864360.2017.1419649
47.
Stone
,
B.
,
Salmon
,
J. L.
,
Hepworth
,
A. I.
,
Red
,
E.
,
Killian
,
M.
,
La
,
A.
,
Pedersen
,
A.
, and
Jones
,
T.
,
2017
, “
Methods for Determining the Optimal Number of Simultaneous Contributors for Multi-User CAD Parts
,”
Comput. Aided Des. Appl.
,
14
(
5
), pp.
610
621
. 10.1080/16864360.2016.1273578
48.
Phadnis
,
V. S.
,
Leonardo
,
K. A.
,
Wallace
,
D. R.
, and
Olechowski
,
A. L.
,
2019
, “
An Exploratory Study Comparing CAD Tools and Working Styles for Implementing Design Changes
,”
ICED19 22nd International Conference on Engineering Design
,
Delft, Netherlands
, pp.
1383
1392
.
49.
Wynn
,
D. C.
, and
Eckert
,
C. M.
,
2017
,
Perspectives on Iteration in Design and Development
,
Springer
,
London
,
28
(
2
).
50.
Nguyen
,
P.
,
Nguyen
,
T. A.
, and
Zeng
,
Y.
,
2018
, “
Segmentation of Design Protocol Using EEG
,”
Artif. Intell. Eng. Des. Anal. Manuf. AIEDAM
,
33
(
1
), pp.
11
23
.
51.
Mcduff
,
D.
,
El Kaliouby
,
R.
,
Senechal
,
T.
,
Amr
,
M.
,
Cohn
,
J. F.
, and
Picard
,
R.
,
2013
, “
Affectiva-MIT Facial Expression Dataset (AM-FED): Naturalistic and Spontaneous Facial Expressions Collected in-the-Wild
,”
Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops
,
Portland, OR
,
June 23–28
.
52.
Kulke
,
L.
,
Feyerabend
,
D.
, and
Schacht
,
A.
,
2020
, “
A Comparison of the Affectiva iMotions Facial Expression Analysis Software With EMG for Identifying Facial Expressions of Emotion
,”
Front. Psych.
,
11
, pp.
1
10
. 10.3389/fpsyg.2020.00329
53.
Qu
,
F.
,
Wang
,
S. J.
,
Yan
,
W. J.
, and
Fu
,
X.
,
2016
, “
CAS(ME)2: A Database of Spontaneous Macro-Expressions and Micro-Expressions
,”
International Conference on Human-Computer Interaction
,
Toronto, Canada
, pp.
48
59
.
54.
Lopez-Rincon
,
A.
,
2019
, “
Emotion Recognition Using Facial Expressions in Children Using the NAO Robot
,”
CONIELECOMP 2019—2019 International Conference on Electronics, Communications and Computers
,
Puebla, Mexico
, pp.
146
153
.
55.
Spaulding
,
S.
, and
Breazeal
,
C.
,
2019
, “
Frustratingly Easy Personalization for Real-Time Affect Interpretation of Facial Expression
,”
2019 8th International Conference on Affective Computing and Intelligent Interaction, ACII 2019
, pp.
531
537
.
56.
Elfenbein
,
H. A.
,
2007
, “
Emotion in Organizations: A Review and Theoretical Integration
,”
Acad. Manage. Annals
,
1
(
1
), pp.
315
386
. 10.5465/078559812
57.
Coburn
,
J. Q.
,
Salmon
,
J. L.
, and
Freeman
,
I.
,
2018
, “
Effectiveness of an Immersive Virtual Environment for Collaboration With Gesture Support Using Low-Cost Hardware
,”
ASME J. Mech. Des.
,
140
(
4
), p.
042001
. 10.1115/1.4039006
58.
Ruzzante
,
S. W.
, and
Bilton
,
A. M.
,
2018
, “
Agricultural Technology in the Developing World: A Meta-Analysis of the Adoption Literature
,”
Proceedings of the ASME 2018 International Design Engineering Technical Conferences and Computers and Information in Engineering Conference IDETC/CIE 2018
,
Quebec City, Canada
,
Aug. 26–29
.
59.
Moen
,
E. L.
,
Fricano-Kugler
,
C. J.
,
Luikart
,
B. W.
, and
O’Malley
,
A. J.
,
2016
, “
Analyzing Clustered Data: Why and How to Account for Multiple Observations Nested Within a Study Participant?
,”
PLoS One
,
11
(
1
), pp.
1
17
. 10.1371/journal.pone.0146721
60.
Grandey
,
A.
,
2008
,
Handbook of Organizational Behavior
,
J.
Barling
, and
C.
Cooper
, eds.,
Sage
,
London
.
61.
Pourtois
,
G.
,
Schettino
,
A.
, and
Vuilleumier
,
P.
,
2013
, “
Brain Mechanisms for Emotional Influences on Perception and Attention: What is Magic and What is Not
,”
Biol. Psychol.
,
92
(
3
), pp.
492
512
. 10.1016/j.biopsycho.2012.02.007
62.
Oh
,
J.
, and
Sundar
,
S. S.
,
2015
, “
How Does Interactivity Persuade ? An Experimental Test of Interactivity on Cognitive Absorption, Elaboration, and Attitudes
,”
J. Commun.
,
65
(
2
), pp.
213
236
. 10.1111/jcom.12147
63.
Sundar
,
S. S.
,
Bellur
,
S.
,
Oh
,
J.
,
Jia
,
H.
, and
Kim
,
H. S.
,
2016
, “
Theoretical Importance of Contingency in Human-Computer Interaction: Effects of Message Interactivity on User Engagement
,”
Commun. Res.
,
43
(
5
), pp.
595
625
. 10.1177/0093650214534962
64.
Häggman
,
A.
,
Tsai
,
G.
,
Elsen
,
C.
,
Honda
,
T.
, and
Yang
,
M. C.
,
2015
, “
Connections Between the Design Tool, Design Attributes, and User Preferences in Early Stage Design
,”
ASME J. Mech. Des.
,
137
(
7
), p.
071408
. 10.1115/1.4030181
65.
Robertson
,
B. F.
, and
Radcliffe
,
D. F.
,
2009
, “
Impact of CAD Tools on Creative Problem Solving in Engineering Design
,”
CAD Comput. Aided Des.
,
41
(
3
), pp.
136
146
. 10.1016/j.cad.2008.06.007
66.
Fixson
,
S. K.
, and
Marion
,
T. J.
,
2012
, “
Back-Loading: A Potential Side Effect of Employing Digital Design Tools in New Product Development
,”
J. Prod. Innov. Manage.
,
29
(
S1
), pp.
140
156
. 10.1111/j.1540-5885.2012.00959.x
67.
Schon
,
D. A.
,
2016
,
The Reflective Practitioner: How Professionals Think in Action
,
Routledge
,
London
.
68.
Buolamwini
,
J.
, and
Gebru
,
T.
,
2018
, “
Gender Shades: Intersectional Accuracy Disparities in Commercial Gender Classification
,”
Proceedings of the 1st Conference on Fairness, Accountability and Transparency, PMLR
,
New York
,
Feb. 23–24
.
69.
Lopez
,
C.
, and
Tucker
,
C.
,
2019
, “Mining Facial Keypoint Data: The Quest Toward Personalized Engineering Applications,”
Emotional Engineering
, Vol.
7
,
S.
Fukuda
, ed.,
Springer
,
Cham, Switzerland
, pp.
97
112
.
70.
Chen
,
M.
,
Anderson
,
J. R.
, and
Sohn
,
M.-H.
,
2001
, “
What Can a Mouse Cursor Tell Us More? Correlation of Eye/Mouse Movements on Web Browsing
,”
CHI'01
,
Seattle, WA
, CHI,pp.
281
282
.
71.
Dering
,
M. L.
,
Tucker
,
C. S.
, and
Kumara
,
S.
,
2018
, “
An Unsupervised Machine Learning Approach to Assessing Designer Performance During Physical Prototyping
,”
ASME J. Comput. Inf. Sci. Eng.
,
18
(
1
), p.
011002
. 10.1115/1.4037434
72.
Bonebright
,
D. A.
,
2010
, “
40 Years of Storming: A Historical Review of Tuckman’s Model of Small Group Development
,”
Hum. Resour. Dev. Int.
,
13
(
1
), pp.
111
120
. 10.1080/13678861003589099
73.
Mcstay
,
A.
,
2020
, “
Emotional AI, Soft Biometrics and the Surveillance of Emotional Life: An Unusual Consensus on Privacy
,”
Big Data Soc.
,
7
(
1
), pp.
1
12
. 10.1177/2053951720904386
You do not currently have access to this content.