Abstract

Managing the design process of teams has been shown to considerably improve problem-solving behaviors and resulting final outcomes. Automating this activity presents significant opportunities in delivering interventions that dynamically adapt to the state of a team in order to reap the most impact. In this work, an artificial intelligence (AI) agent is created to manage the design process of engineering teams in real time, tracking features of teams’ actions and communications during a complex design and path-planning task in multidisciplinary teams. Teams are also placed under the guidance of human process managers for comparison. Regarding outcomes, teams perform equally as well under both types of management, with trends toward even superior performance from the AI-managed teams. The managers’ intervention strategies and team perceptions of those strategies are also explored, illuminating some intriguing similarities. Both the AI and human process managers focus largely on communication-based interventions, though differences start to emerge in the distribution of interventions across team roles. Furthermore, team members perceive the interventions from both the AI and human manager as equally relevant and helpful, and believe the AI agent to be just as sensitive to the needs of the team. Thus, the overall results show that the AI manager agent introduced in this work is able to match the capabilities of humans, showing potential in automating the management of a complex design process.

References

1.
Paulus
,
P.
,
Dzindolet
,
B.
, and
Kohn
,
M.N. W.
,
2012
, “
Collaborative Creativity – Group Creativity and Team Innovation
,”
Handbook of Organizational Creativity
, pp.
327
357
.
2.
Seat
,
E.
, and
Lord
,
S. M.
,
1998
, “
Enabling Effective Engineering Teams: A Program for Teaching Interaction Skills
,”
FIE ‘98. 28th Annual Frontiers in Education Conference. Moving From “Teacher-Centered” to “Learner-Centered” Education. Conference Proceedings (Cat. No. 98CH36214)
,
Tempe, AZ
,
Nov. 4–7
, IEEE, pp.
246
251
.
3.
Horwitz
,
S. K.
, and
Horwitz
,
I. B.
,
2007
, “
The Effects of Team Diversity on Team Outcomes: A Meta-Analytic Review of Team Demography
,”
J. Manage.
,
33
(
6
), pp.
987
1015
.
4.
Dahlin
,
K. B.
,
Weingart
,
L. R.
, and
Hinds
,
P. J.
,
2005
, “
Team Diversity and Information Use
,”
Acad. Manag. J.
,
48
(
6
), pp.
1107
1123
.
5.
Dong
,
A.
,
Hill
,
A. W.
, and
Agogino
,
A. M.
,
2004
, “
A Document Analysis Method for Characterizing Design Team Performance
,”
ASME J. Mech. Des.
,
126
(
3
), pp.
378
385
.
6.
McComb
,
C.
,
Cagan
,
J.
, and
Kotovsky
,
K.
,
2017
, “
Optimizing Design Teams Based on Problem Properties: Computational Team Simulations and an Applied Empirical Test
,”
ASME J. Mech. Des.
,
139
(
4
), p.
041101
.
7.
Tribelsky
,
E.
, and
Sacks
,
R.
,
2011
, “
An Empirical Study of Information Flows in Multidisciplinary Civil Engineering Design Teams Using Lean Measures
,”
Archit. Eng. Des. Manag.
,
7
(
2
), p.
85
101
.
8.
Karau
,
S. J.
, and
Wilhau
,
A. J.
,
2019
,
Individual Motivation Within Groups: Social Loafing and Motivation Gains in Work, Academic, and Sports Teams
,
Academic Press
,
Cambridge, MA
, pp.
3
51
.
9.
Gardner
,
H. K.
,
2012
, “
Performance Pressure as a Double-Edged Sword
,”
Adm. Sci. Q.
,
57
(
1
), pp.
1
46
.
10.
Cohen
,
G. P.
,
1993
, “
The Virtual Design Team: An Information-Processing Model of Design Team Management
,”
Doctoral dissertation
,
Stanford University
.
11.
Kunz
,
J. C.
,
Cohen
,
G. P.
, and
Levitt
,
R. E.
,
1993
, “
Modeling Effects of Organizational Structure and Communication Tools on Design Team Productivity
,”
AI and Theories of Groups and Organizations: Conceptual and Empirical Research. Papers from the 1993 AAAI Workshop Technical Report WS-93-03
,
Menlo Park, CA
.
12.
De Montjoye
,
Y. A.
,
Stopczynski
,
A.
,
Shmueli
,
E.
,
Pentland
,
A.
, and
Lehmann
,
S.
,
2014
, “
The Strength of the Strongest Ties in Collaborative Problem Solving
,”
Sci. Rep.
,
4
(
1
), pp.
1
6
.
13.
den Otter
,
A.
, and
Emmitt
,
S.
,
2008
, “
Design Team Communication and Design Task Complexity: The Preference for Dialogues
,”
Archit. Eng. Des. Manag.
,
4
(
2
), pp.
121
129
.
14.
Senescu
,
R. R.
,
Aranda-Mena
,
G.
, and
Haymaker
,
J. R.
,
2013
, “
Relationships Between Project Complexity and Communication
,”
J. Manag. Eng.
,
29
(
2
), pp.
183
197
.
15.
Heisig
,
P.
,
Clarkson
,
P. J.
, and
Vajna
,
S.
,
2010
,
Modelling and Management of Engineering Processes
,
Springer
,
New York
.
16.
Senescu
,
R. R.
, and
Haymaker
,
J. R.
,
2009
, “
Specifications for a Social and Technical Environment for Improving Design Process Communication
,”
Proceedings of the 26th International Conference on IT in Construction
,
Istanbul, Turkey
,
Oct. 1–3
, pp.
227
237
.
17.
Zurita
,
N. F. S.
, and
Tumer
,
I. Y.
,
2017
, “
A Survey: Towards Understanding Emergent Behavior in Complex Engineered Systems
,”
Proceedings of the ASME Design Engineering Technical Conference.
,
Cleveland, OH
,
Aug. 6–9
.
18.
Song
,
B.
,
Soria Zurita
,
N. F.
,
Zhang
,
G.
,
Stump
,
G.
,
Balon
,
C.
,
Miller
,
S. W.
,
Yukish
,
M.
,
Cagan
,
J.
, and
McComb
,
C.
,
2020
, “
Toward Hybrid Teams: A Platform to Understand Human-Computer Collaboration During the Design of Complex Engineered Systems
,”
Proceedings of the Design Society: DESIGN Conference
,
Cavtat, Croatia
,
Oct. 26–29
.
19.
Koch
,
J.
, and
Paris-Saclay
,
I.
,
2017
, “
Design Implications for Designing With a Collaborative AI
,”
AAAI Spring Symposium Series.
,
Palo Alto, CA
,
Mar. 27–29
.
20.
Camburn
,
B.
,
Arlitt
,
R.
,
Anderson
,
D.
,
Sanaei
,
R.
,
Raviselam
,
S.
,
Jensen
,
D.
, and
Wood
,
K. L.
,
2020
, “
Computer-Aided Mind Map Generation via Crowdsourcing and Machine Learning
,”
Res. Eng. Des.
,
31
(
4
), pp.
383
409
.
21.
Camburn
,
B.
,
He
,
Y.
,
Raviselvam
,
S.
,
Luo
,
J.
, and
Wood
,
K.
,
2020
, “
Machine Learning-Based Design Concept Evaluation
,”
ASME J. Mech. Des.
,
142
(
3
), p.
031113
.
22.
Dering
,
M. L.
,
Tucker
,
C. S.
, and
Kumara
,
S.
,
2018
, “
An Unsupervised Machine Learning Approach to Assessing Designer Performance During Physical Prototyping
,”
ASME J. Comput. Inf. Sci. Eng.
,
18
(
1
), p.
011002
.
23.
Williams
,
G.
,
Meisel
,
N. A.
,
Simpson
,
T. W.
, and
McComb
,
C.
,
2019
, “
Design Repository Effectiveness for 3D Convolutional Neural Networks: Application to Additive Manufacturing
,”
ASME J. Mech. Des. Trans.
,
141
(
11
), p.
111701
.
24.
Jin
,
Y.
,
Levitt
,
R. E.
,
Christiansen
,
T. R.
, and
Kunz
,
J. C.
,
1995
, “
The Virtual Design Team: Modeling Organizational Behavior of Concurrent Design Teams
,”
AI EDAM
,
9
(
2
), pp.
145
158
.
25.
Zhang
,
G.
,
Raina
,
A.
,
Cagan
,
J.
, and
McComb
,
C.
,
2021
, “
A Cautionary Tale About the Impact of AI on Human Design Teams
,”
Des. Stud.
,
72
, p.
100990
.
26.
Rao
,
S. S.
,
Nahm
,
A.
,
Shi
,
Z.
,
Deng
,
X.
, and
Syamil
,
A.
,
1999
, “
Artificial Intelligence and Expert Systems Applications in New Product Development—A Survey
,”
J. Intell. Manuf.
,
10
(
3
), pp.
231
244
.
27.
Raina
,
A.
,
Cagan
,
J.
, and
McComb
,
C.
,
2019
, “
Transferring Design Strategies From Human to Computer and Across Design Problems
,”
ASME J. Mech. Des.
,
141
(
11
), p.
114501
.
28.
Maier
,
T.
,
Zurita
,
N. F. S.
,
Starkey
,
E.
,
Spillane
,
D.
,
Menold
,
J.
, and
McComb
,
C.
,
2020
, “
Analyzing the Characteristics of Cognitive-Assistant-Facilitated Ideation Groups
,”
Proceedings of the ASME Design Engineering Technical Conference
,
Virtual, Online
,
Aug. 17–19
.
29.
Maier
,
T.
,
Abdullah
,
S.
,
McComb
,
C.
, and
Menold
,
J.
,
2021
, “
A Query Conundrum: The Mental Challenges of Using a Cognitive Assistant
,”
SN Comput. Sci.
,
2
(
3
), p.
194
.
30.
Gyory
,
J. T.
,
Cagan
,
J.
, and
Kotovsky
,
K.
,
2019
, “
Are You Better off Alone? Mitigating the Underperformance of Engineering Teams During Conceptual Design Through Adaptive Process Management
,”
Res. Eng. Des.
,
30
(
1
), pp.
85
102
.
31.
Goucher-Lambert
,
K.
,
Gyory
,
J. T.
,
Kotovsky
,
K.
, and
Cagan
,
J.
,
2020
, “
Adaptive Inspirational Design Stimuli: Using Design Output to Computationally Search for Stimuli That Impact Concept Generation
,”
ASME J. Mech. Des.
,
142
(
9
), p.
091401
.
32.
Landauer
,
T. K.
,
Foltz
,
P. W.
, and
Laham
,
D.
,
1998
, “
An Introduction to Latent Semantic Analysis
,”
Discourse Process.
,
25
(
2–3
), pp.
259
284
.
33.
Gyory
,
J.
,
Kotovsky
,
K.
, and
Cagan
,
J.
,
2021
, “
The Influence of Process Management: Uncovering the Impact of Real-Time Managerial Interventions via a Topic Modeling Approach
,”
ASME J. Mech. Des.
,
143
(
11
), p.
111401
.
34.
HyFormTM GitHub
”. https://github.com/hyform/drone-testbed-server/releases/tag/2021-March-v2, Accessed April 23, 2021.
35.
Zhang
,
G.
,
Soria Zurita
,
N. F.
,
Stump
,
G.
,
Song
,
B.
,
Cagan
,
J.
, and
McComb
,
C.
,
2021
, “
Data on the Design and Operation of Drones by Both Individuals and Teams
,”
Data Br.
,
36
, pp.
1
11
.
36.
Gero
,
J. S.
,
Jiang
,
H.
,
Dobolyi
,
K.
,
Bellows
,
B.
, and
Smythwood
,
M.
,
2015
, “
How Do Interruptions During Designing Affect Design Cognition
,”
Design Computing and Cognition'14
,
London, UK
,
June 23–25
.
37.
Cades
,
D. M.
,
Boehm-Davis
,
D. A.
,
Trafton
,
J. G.
, and
Monk
,
C. A.
,
2011
, “
Mitigating Disruptive Effects of Interruptions Through Training: What Needs to be Practiced?
,”
J. Exp. Psychol. Appl.
,
17
(
2
), pp.
97
109
.
38.
Hess
,
S. M.
, and
Detweiler
,
M. C.
,
1994
, “
Training to Reduce the Disruptive Effects of Interruptions
,”
Proc Human Factors Ergonom. Soc. Annual Meet.
,
38
(
18
), pp.
1173
1177
.
39.
Sio
,
U. N.
,
Kotovsky
,
K.
, and
Cagan
,
J.
,
2017
, “
Interrupted: The Roles of Distributed Effort and Incubation in Preventing Fixation and Generating Problem Solutions
,”
Mem. Cogn.
,
45
(
4
), pp.
553
565
.
40.
Lock
,
S.
,
2013
, “
Anachronistic Intervention: Performative Collaborative Design in the Wild
,”
CoDesign
,
9
(
1
), pp.
17
36
.
You do not currently have access to this content.