Abstract

High-performing design teams are characterized by their ability to maintain performance across a variety of problem types. This is often referred to as robustness, and is usually achieved through careful management of team processes. However, there exists an opportunity to design teams that are likely to be inherently robust by addressing and embracing the individual variability of team members. Cognitive style provides an avenue by which we can compose robust teams based on the problem-solving approach of the individual. In this work, we used the KAI agent-based organizational optimization model (KABOOM) to evaluate the effects of team composition and team structure on the robustness of overall team performance. Teams of homogeneous and heterogeneous KAI styles were tasked to solve a variety of different abstract design problems and evaluated based on their performance with and without sub-teams. Results indicate that there is a significant difference in the distribution of aggregate scores for homogeneous and heterogeneous teams without sub-teams, and heterogeneous teams may be more robust. Sub-teams were found to significantly increase the overall median score and robustness for some teams.

Introduction

High-performance design teams must achieve a level of robustness—the ability to perform well and consistently across a variety of complex design tasks. In practice, robustness is often achieved through careful management of the design process. The literature emphasizes the importance of team management to reduce failures [1,2], and team management is often regarded as the backbone structure of any organization [3]. However, there exists a potential to carefully compose and structure teams so that they are more likely to be inherently robust. Cognitive style, the preference for different levels of structure in problem-solving [4], presents a significant opportunity to form robust teams based on the individual differences of team members. In this work, we examine the robustness of teams with different structures and cognitive style compositions through a computational simulation study.

Cognitive style, particularly as defined in Kirton’s Adaption-Innovation (A-I) theory, directly addresses dimensions of problem-solving that are relevant for engineering and design [4]. Cognitive style refers to the natural and preferred way of individual processes and organizes information [5] which in turn influences the problem-solving process [6]. The A-I theory is thus concerned with how people solve problems. This is often contrasted with Kirton’s definition of cognitive level, which refers to an individual’s capacity for problem solving and creative behavior and is often measured with intelligence tests [7]. The cognitive level is independent of cognitive style and representative of the potential capacity to solve problems. While varying cognitive levels are necessary for large, complex problems [8], the correct cognitive level is often clearly related to the task at hand. Differences in cognitive style, however, produce distinct patterns of behavior that interact when people solve problems collaboratively. Both cognitive style and cognitive level influence one’s ability to solve problems. Therefore, cognitive style from an A-I theory perspective is a potential avenue of which to study robustness with respect to team composition and structure. Differences in cognitive style produce distinct patterns of behavior which are important, when people solve problems collaboratively, independent of the cognitive level of the individual.

To measure the cognitive style in the problem-solving context, Kirton developed the Adaption-Innovation Inventory (KAI). The KAI places one’s cognitive style on a numerical bi-polar continuum, ranging from highly adaptive to highly innovative, with a theoretical range of 32–160 [9]. More “adaptive” problem-solvers prefer to work with more structure, prefer known solutions, and make incremental changes while seeking to maintain group cohesion [8]. On the other end of the spectrum, more “innovative” problem-solvers are more tolerant of a loosely-guided structure, cutting across paradigms and the existing structure to solve problems “differently”, with less concern for group consensus [8]. Kirton referred to innovators as those who have a tendency to “overhaul the entire work process”, approaching tasks from unexpected angles and pursuing radically different solutions with less regard to existing structures [10,11]. This continuum is rooted in the axiom that wanting “to do things better” (i.e., the adaptive end of the spectrum) and wanting “to do things differently” (the innovative end) are equally creative, although different styles may be desirable depending on the situation [12]. Through multiple validation studies, Kirton also developed three sub-scores that are based on the sub-factors of cognitive style. The sufficiency of originality (SO) sub-factor highlights the differences between individuals and their preferred ways of generating and offering ideas. The efficiency (E) sub-factor reflects an individual’s preferred methods for gathering and organizing ideas. The rule/group conformity sub-factor (R/G) reflects differences in the ways individuals manage the personal and impersonal structures in which their problem solving occurs. Earlier work has leveraged the A-I theory to determine how the KAI score affects individuals’ ability to problem solve when given adaptively or innovatively framed design problems [13].

Kirton’s theory has been routinely applied in the earlier literature to examine the cognitive processes of collaborative teams. KAI has been used to determine the cognitive styles of engineering and management students in project teams, which showed correlations between positive and negative aspects of project team experiences [14]. When studying the relationship between cognitive style and team interactions during concept generation, the results suggest that more innovative teams tend to have interactions with a higher degree of integration between topics [15]. Another study examining team success and cognitive diversity using the KAI measure to examine team cognitive styles found that teams had higher levels of success when tasks were coordinated with team members’ KAI scores (e.g., a more adaptive task with a more adaptive sub-team) [16]. It was also found that homogeneous teams were successful due to enhanced inter-team communication, whereas heterogeneous teams fell as a result of unresolved cognitive gaps [16]. In terms of creative fluency, another researcher found that heterogeneous teams outperformed homogeneous teams in terms of creative output [17]. These studies establish KAI as a particularly useful framework for assessing the impact of cognitive style on performance within design teams.

Two important dimensions for describing a team are composition and structure. Team composition is often viewed in terms of homogeneity or heterogeneity. While there is recognition that heterogeneity in teams benefits with creativity in problem-solving, homogeneous teams may be better at the exploitation of existing knowledge [18]. The team structure is often examined through the degree of hierarchy within the team. Teams often form implicit hierarchal structures (e.g., sub-teams) once they reach a certain size [19]. Subgrouping has been shown to enhance creativity within virtual and collocated design teams [20]. Furthermore, some studies indicate that teams achieve better performance with frequent interaction and communication across sub-teams [21,22].

As teams become more interdisciplinary and develop more context structures, selecting the right members becomes more critical [23]. However, the research on optimal team formation strategies is limited. Studies on design teams in vivo are often expensive and applicable only to specific contexts within their scope [24]. Therefore, this work utilizes agent-based modeling (ABM), a type of computational simulation, instead of conducting traditional human subjects studies. This makes it possible to rapidly vary team composition and structure to examine robustness while carefully controlling specific variables, which is challenging in human subjects studies. Agent-based modeling to simulate teamwork has been undertaken frequently in the literature [2528] and used to understand human systems in the design [29]. One study, in particular, used ABM to examine how team member selection and diversity affect team performance in various economic conditions [30]. Using homogeneous, heterogeneous, and interdependence-based strategies to form project teams, the model explored the relative effects of different team member orientations on firm performance or team profit. This study showed that while the difference in profit earned from homogeneous and heterogeneous teams is not significant, heterogeneous teams can enhance the overall firm performance in certain conditions. Recently, some studies have also used ABM to model the social and emotional aspects of team problem solving [31], while others have used ABM to model the effects of stress and motivation on team performance [32].

In this work, we study simulated cognitive styles informed by the KAI measure. Adaption-Innovation theory has been directly used in the KAI agent-based organizational optimization model (KABOOM) to simulate the cognitive style in problem-solving teams. KABOOM simulates team problem-solving by executing a modified multi-agent simulated annealing algorithm [33]. In KABOOM, a team of agents begins a stochastic search of the solution space and gradually transitions to a downhill search to refine a solution. Simulated annealing has been validated earlier as an effective model of human problem-solving behavior [3436]. The three main modifications from a basic simulated annealing algorithm include agents possessing unique cognitive styles to modify their exploration of a solution space, teams of agents who specialize in decomposing a problem into sub-problems, and agents communicating to share solutions in pair-wise and team-wise meetings [11]. Autonomous agents with various cognitive styles interact to solve a diverse set of problems and maximize an objective function, indicating performance in KABOOM. KABOOM simulates cognitive style by assigning an overall KAI score and related sub-scores to an agent. The overall KAI score affects how agents perceive the solution quality and their exploration of the solution space, where more adaptive agents take smaller steps throughout the simulation and more innovative agents take larger steps. The KAI sub-scores for each agent further represent the dimensions of cognitive style and model more specific aspects of human problem-solving behavior and solution generation. While KABOOM assumes that agents can evaluate the quality of their solutions using the objective function, their perception of the solution quality is affected by their cognitive styles [11].

In prior work, KABOOM has been used to simulate the performance of teams and agents with homogeneous and heterogeneous cognitive styles on two contextualized design problems [37]. For the first design problem, it was observed that changing the cognitive style of the homogeneous teams did not impact performance. For the second design problem, more innovative homogeneous teams outperformed the adaptive and mid-range teams. Overall, results demonstrated that some cognitive styles were more effective, depending on the problem. In another study, homogeneous teams of more adaptive, more innovative, and mid-range styles were tested on a set of 25 design problems to determine the optimal cognitive style for each problem. Larger search spaces favored more innovative cognitive styles, while smaller search spaces favored more adaptive styles [11]. When applied in different scenarios, KABOOM accurately models the trends found in A-I theory. For example, KABOOM emulates the communication styles of teams arranged in different network structures [38] and has exemplified the performance of specialized software development teams [39]. Further, it has been shown to represent trends in communication [40,41] and specialization [42] in human problem-solving.

We use the python implementation of the KABOOM framework in this work to generate teams of computational agents with varying simulated cognitive styles. This approach will evaluate how the agent composition and structure impacts robustness. By using KABOOM to study teams, we can rapidly simulate approximately 5000 design teams in this work, giving insight into potential strategies for team organization. Replicating this experiment using human subjects would be time and resource-intensive for both academic and industry research. KABOOM also enables detailed control over individual agents’ cognitive styles and over team composition—a level of control that would be challenging to achieve in human subjects research. By simulating thousands of teams, organizations will be able to predict the performance of the team before expending resources to assemble one. This leads to significant savings in cost, time, and productivity.

At the intersection of cognitive style and agent-based modeling, this research aims to determine how team composition and structure impact the robustness of engineering design teams. In prior research, it was found that varying the composition and hierarchal structure of design teams can affect their collaboration and problem-solving performance [9,14,16,18,19,22]. Therefore, we introduce two research questions that seek to understand how team robustness is affected by variations in cognitive style composition and structural hierarchy, respectively:

  1. How does the robustness of team performance compare when homogeneous teams and heterogeneous teams solve the same set of design problems?

  2. How does the addition of sub-teams affect the robustness of team performance in comparison to teams without sub-teams?

Methods

For this study, we used the KABOOM framework2 to evaluate the impact of team composition (varying homogeneous and heterogeneous team types) and team structure (either with or without sub-teams) on robustness. As teams tend to naturally develop some degree of internal structure when larger than six members, we construct each team with six agents each [19]. Each team was tasked with solving the same set of 25 diverse design problems. By examining the distribution of performance across these problems, it becomes possible to assess how team structure and composition impact robustness. Every team is simulated for 300 iterations. Agents communicate with pairwise interactions occurring organically at a 20% frequency with each iteration. Agents that decide to communicate are paired and attempt to share their solutions.

The Problem Set.

This paper implements a 12-dimensional mathematical objective function that can be tuned and scaled in predictable ways as a representation of the variability in design problem characteristics [11]. It is represented by a scalar objective function f(x) of n dimensions which agents are tasked with minimizing. Specifically, the objective function is a summation of a quadratic term and a sinusoidal term in the form:
f(x)=i=1nαcos(ωxiβ)C(xiβ)2for0.5xi0.5
(1)
where α is the oscillation amplitude of the sinusoid, β is the scaling parameter that scales the size of the solution space, ω affects the wavelength of the sinusoid, and C is a scaling constant that impacts the relative size of the quadratic. The shape of this function can be varied in two ways: (1) by scaling the independent variables in all dimensions using the scaling parameter, β, and (2) by scaling the oscillation amplitude of the sinusoid, α. The β, parameter affects the size of the search space which scales agents' step sizes dependent on their cognitive style, while the α parameter affects the amplitude of the sinusoid, indicative of the relative quality of solutions in the search space. By varying these parameters, we create 25 unique design problems that may favor different cognitive styles. Each team solves these 25 design problems for eight repetitions. We do not attempt to generalize directly to specific classes of problems, as the functional form that we use here is very specific. However, this specific form is sufficient to induce wide variability across potential problem-solving approaches [11,37]. Specifically, the set of problems used here was shown in prior work [11] to have sufficient variability to be optimally solved by the full range of adaptive and innovative homogeneous teams. Therefore, this set of problems is ideal for studying robustness in the present work.

Team Composition.

In this work, the impact of team composition is examined through a variety of heterogeneous or homogeneous team compositions. In practice, KAI scores are addressed as an overall score, so sub-scores are not controlled for in this simulation. Homogeneous teams are composed of agents with the same overall KAI score, whereas heterogeneous teams are composed of agents with differing KAI scores. The simulated teams were composed using three different strategies. In the first composition strategy, homogeneous teams, all six agents in the team shared the same KAI score; seven different homogeneous team types were instantiated, linearly spaced between a score of 60 and 140. The second composition strategy composed of heterogeneous teams. These teams also included six agents, produced by sampling from the uniform distribution of the complete range of KAI scores observed in the population. This selection was iteratively narrowed to more mid-range scores, totaling five heterogeneous team types.

In addition to the teams mentioned above, we also simulated a single team with an organic composition. We compare the team composition strategies mentioned above to a team with an organic composition. The organic composition was generated by randomly selecting individuals from a virtual population corresponding to the normal distribution of KAI scores observed in the general human population (mean = 92.93, STD = 18.20) gathered in the earlier research [43]. The organic team-best represents a team selected without an overt strategy. Therefore, organic composition serves as a baseline against which to compare other approaches.

Team Structure.

In addition to team composition, this work explores the impact of team structure (with or without sub-teams). Each problem that teams were tasked with solving contains 12 variables that can be decomposed by assigning subsets of the variables to different sub-teams, where each sub-team has unique control over their assigned variables, and assignments are not changed for the duration of the simulation [37]. We assess the impact of this team structure on both homogeneous and heterogeneous team compositions. In flat teams (those without sub-teams), the team collectively works on all 12 variables of the problem. In hierarchal (those with sub-teams) teams, each team was split into three sub-teams per team, where each sub-team controls a subset of all the dimensions in the problem. Three sub-teams per team were chosen as a representative value to emphasize problem specialization.

As teams often converge to analyze and evaluate solution ideas before moving forward in the presence of complex design problems [44], teams with sub-teams experienced team meetings. Team meetings occur every 30 iterations and result in all agents on the team converging to one solution. First, the team creates an aggregate solution from the specialized sub-teams. Each sub-team finds the best solution of any of its agents, then the team forms an aggregate solution of from the specialized dimensions of that sub-team’s best solution [11].

Quantifying Robustness.

For every combination of team composition and team structure, 200 independent team simulations were performed (25 problems with eight repetitions each). At the end of each of these simulations, the team’s performance is the solution value of the best solution any agent has had at any time during the simulation, quantified as a numerical value. As the teams are solving a minimization problem in the simulation, lower numerical values indicate a better performance. We examine three values computed from the set of solution values for each combination. These include the median value, the lower quartile or 25th percentile value, and the worst value (after removing outliers). The median value indicates the median performance of the team. The 25th percentile value is calculated as the upper quartile (75th) value but presented as the lower quartile in the simulation to represent poorer performance in our minimization function. The worst score for each team represents the maximum score, excluding outliers. Outliers represent any value that exceeds the value calculated from the interquartile range multiplied by 1.5 [45]. In such cases, the latter value was presented as the worst score. In this work, high robustness in a team indicates their ability to reduce worst-case outcomes, achieving performance across problems with less detrimental variability. High robustness in a team indicates their propensity for performing consistently nearer to their median performance. A broad spread across these values indicates lower robustness (high dependence of solution quality on problem type), while a low spread indicates higher robustness (low dependence on problem type).

In addition to these values, the median, 25th percentile, and worst performance of the organic team are presented as a baseline against the strategized teams.

Results

In this section, we discuss the robustness and median performance of simulated teams as a function of hierarchy and cognitive style. A Shapiro–Wilk test to assess normality indicated a significant departure from normality in our data. Therefore, multiple Mann–Whitney U tests were used to quantify differences in the objective function values between the various team compositions and structures. The Mann–Whitney U is a nonparametric test that can be used to determine if there are differences between two groups on a continuous scale [46].

The results of the strategized and organic team are presented in the figures below. Each figure shows the median, 25th percentile, and worst-case performance for each team composition. The error bars represent median absolute deviation, or the measure of variability around each data point. Horizontal dashed lines show the median, 25th percentile, and worst-case performance of an organically composed team as a baseline comparison. The median absolute deviation for the organic teams is not presented in the plots. As the agents are tasked with minimization problems, lower objective function values indicate better performance. We compare the trends with respect to robustness and overall median performance for each strategy and present three comparisons in the subsections. The first subsection addresses our first research question, how does the robustness of team performance compare when homogeneous teams and heterogeneous teams solve the same set of design problems? and presents the results of the homogeneous flat and heterogeneous flat teams. The second subsection addresses our second research question, how does the addition of sub-teams affect the robustness of team performance in comparison to teams without sub-teams presents the comparison between the flat and hierarchal teams.

Homogeneous and Heterogeneous Flat Teams.

The results of our homogeneous and heterogeneous flat teams are presented in Fig. 1.

Fig. 1
Results of the (a) homogeneous flat team and (b) heterogeneous flat team. Error bars represent median absolute deviation.
Fig. 1
Results of the (a) homogeneous flat team and (b) heterogeneous flat team. Error bars represent median absolute deviation.
Close modal

A Mann–Whitney U test was run to evaluate if there exists a difference in the aggregate results of the performance between homogeneous and heterogeneous flat teams (those without sub-teams). Scores from each team in the homogeneous flat composition were aggregated and compared against the aggregate scores from the heterogeneous flat composition. Statistically significant differences were found between the aggregate values of the two groups U(Nhomogeneous = 1400 Nheterogeneous = 1000) = 596,208, p < 0.001. This indicates that there are significant differences in the solutions outputted by either composition. Upon observation, as KAI scores for the homogeneous teams became more innovative, worst-scores became more severe with higher median absolute deviations. This indicates that as KAI scores increase, teams become less robust. In contrast, the heterogeneous teams performed more consistently than homogeneous teams, with lower variability for the median, 25th percentile, and worse-case scores. This indicates that heterogeneity in teams can help improve robustness.

The trend in the data observed in the homogeneous flat teams illustrates that higher KAI scores were positively correlated to better median performance. However, the most innovative team (KAI 140) saw a decrement in median performance. More research needs to be conducted to determine if median performance continues to decrement as team KAI scores increase past 140. In contrast to median performance, homogeneous flat teams only experienced better 25th percentile and worst-case scores for the more adaptive teams, specifically teams below a KAI of 100. In summary, slightly more innovative teams have a better median performance when faced with various problems compared to adaptive or highly innovative teams but are less robust as they are at a greater risk for outputting poor solutions. As more innovative individuals are known to think in risky and unexpected ways, these results seem to mimic real-life behavior commensurate with the A-I theory [47]. Finally, homogeneous teams showed a high degree of variability in their median performance, depending on the specific cognitive style. However, the performance of all heterogeneous teams aligned closely with that of the baseline organic teams.

Flat Teams Versus Hierarchal Teams.

Figure 2 illustrates the side-by-side results of homogeneous flat teams and homogeneous hierarchal teams. The earlier illustration for the homogeneous flat teams is provided as a comparison to the homogeneous hierarchal teams.

Fig. 2
Results of the (a) homogeneous flat team and (b) homogeneous hierarchal team. Error bars represent median absolute deviation.
Fig. 2
Results of the (a) homogeneous flat team and (b) homogeneous hierarchal team. Error bars represent median absolute deviation.
Close modal

A Mann–Whitney test found that the differences in the distribution of the aggregate scores between homogeneous flat and hierarchal composition strategies were statistically significant U(Nflat = 1400 Nsub-team = 1400) = 882,149, p < 0.001. This indicates that there are significant differences in the solutions outputted by the hierarchal composition. In comparison to the homogeneous flat teams, the addition of sub-teams allowed for a smaller median absolute deviation, providing more consistency and a smaller spread of median, 25th percentile, and worst-case scores, especially for the more innovative teams. For more innovative teams, sub-teams greatly increased performance among the 25th percentile and worst-case scores. Specifically, hierarchal teams with a KAI score of 100 or higher were more robust than the flat teams, as indicated by the smaller spread of scores. Forming sub-teams in homogeneous team composition also helped to increase median performance for every team type except for our most adaptive teams (KAI score = 60), which did not benefit from the formation of sub-teams. Mann–Whitney U tests showed that for certain teams, these differences in median performance were statistically significant. Hierarchal teams with a KAI score of 100 had a significantly better median performance than corresponding flat teams with a KAI score of 100 U(Nflat = 200 Nsub-team = 200) = 15904, p = 0.001. Similarly, hierarchal teams with a KAI score of 113 outperformed a corresponding flat team with a KAI score of 113. Mann–Whitney confirmed that this difference was statistically significant U(Nflat = 200 Nsub-team = 200) = 17294, p = 0.009. The differences in median performance between the remainder of the teams were not found to be statistically significant. This demonstrates that teams that are slightly more innovative benefit significantly in median performance when organized into specialized sub-teams.

The results indicate that forming homogeneous teams with specialized sub-teams can greatly improve robustness, especially for more mid-range and innovative teams. Future work will evaluate if these results remain consistent with varying degrees of specialization. In sum, modifying the organization within the team can have significant implications on robustness and median performance, even if the KAI composition of the teams remains the same. It is also of note that among both composition strategies, the most adaptive teams performed the poorest regardless of team structure.

Figure 3 illustrates the side-by-side results of heterogeneous flat teams and heterogeneous hierarchal teams. The earlier illustration for the heterogeneous flat teams is provided as a comparison to the heterogeneous hierarchal teams.

Fig. 3
Results of the (a) heterogeneous flat team and (b) heterogeneous hierarchal team. Error bars represent median absolute deviation.
Fig. 3
Results of the (a) heterogeneous flat team and (b) heterogeneous hierarchal team. Error bars represent median absolute deviation.
Close modal

While the differences in both robustness and median performance are visually less apparent between the flat and hierarchal compositions, a Mann–Whitney U test showed that there exists a significant difference in the distribution of the aggregate scores U(Nflat = 1000 Nsub-team = 1000) = 426,603, p < 0.0001. When comparing the results of the individual teams, heterogeneous hierarchal teams had significantly better median scores than the heterogeneous flat teams, except for the team pooled from the full distribution of scores (100 ± 40). A Mann–Whitney U test confirmed that these differences were statistically significant (for 100 ± 30 U(Nflat = 200 Nhierarchal = 200) = 878, p < 0.0001, for 100 ± 20 U(Nflat = 200 Nhierarchal = 200) = 787, p < 0.0001, for 100 ± 10 U(Nflat = 200 Nhierarchal = 200) = 673, p < 0.0001, for 100 ± 5 U(Nflat = 200 Nhierarchal = 200) = 733, p < 0.0001). For the team pooled from the full distribution of scores (100 ± 40), implementing sub-teams helped to increase robustness, visually apparent by the smaller spread of scores for the hierarchal team. Clearly, with diverse teams, specialized sub-teams can help increase median performance. Comparison with homogeneous hierarchal teams versus heterogeneous hierarchal teams followed similar trends.

Throughout the simulation study, it becomes apparent that some combinations of team structure and composition achieve high robustness while forfeiting median performance; for instance, highly adaptive homogeneous flat teams. We hypothesize that the highly localized search space exploration from a highly adaptive team is insufficient for solving problems that would benefit from a more global exploration resulting in a lower overall median performance. Homogeneous adaptive teams do not explore radical solutions that could result in poor performance, but they also do not explore the solution space sufficiently to find high-quality solutions [11]. These trends emulate A-I theory, where adaptors tend to “dig deeply” within the solution space and are less likely to explore the breadth of the solution space [9,48]. Other teams achieve good median performance while forfeiting robustness; for instance, highly innovative homogeneous flat teams. We hypothesize that a highly innovative team’s global exploration of the search space is sufficient for all problems, however, larger step sizes result in solutions that are very distinct from one another [11]. This increases the frequency and severity of poorer solutions, resulting in lower robustness. This is consistent with the A-I theory, as innovators are more likely to generate ideas that “jump around” within a solution space [9,48]. This tradeoff between robustness and median performance is potentially important for informing early team composition and structure decisions.

Limitations and Future Work

In this study, we evaluated the effects of team composition and team structure on robustness and median performance through the lens of Kirton’s theory of cognitive style. While cognitive style can be a valuable avenue from which we can use to form strategic teams, other factors also determine team performance, such as skills, proficiency, and personality. In addition, other aspects of cognitive styles, such as coping behavior [4], are omitted in the model to quantitatively assess the direct impact of individual KAI scores in collaboration with a team. Our findings in this work are also limited to the specific model parameters. It is possible that altering the number of agents per team, number of sub-teams, or frequency of team meetings may provide additional insights. While this model validates the behavior of adaptors and innovators described in A-I theory, it is not a comprehensive model of human problem-solving. Rather, this experiment allows us to predict how teams will perform given a variety of design problems with respect to their cognitive style. Future work will seek to validate results with human subjects. With all simulations of complex real-world systems, validation is necessary to truly understand the applicability of the numerical results. Extensive validation of our research outcomes is a subject of future work.

Conclusion

This experiment demonstrates that composing teams based on individual cognitive style may have significant impacts on a team’s ability to be robust given a variety of design problems. It also reveals that teams can be composed based on cognitive style in such a way that they outperform an organic team. Our first research question sought to evaluate the differences in robustness between homogeneous and heterogeneous team compositions. In this experiment, we concluded that homogeneous flat teams begin to forfeit robustness as their cognitive style composition becomes more innovative, while heterogeneous teams maintain robustness across the specific team compositions. Our second research question further examined how decomposing teams into specialized sub-teams further impacts robustness. We found that with the homogeneous teams, specialization can help reduce the severity of worst-case performance and thus increase robustness. Implementing hierarchy in heterogeneous teams also proves advantageous for median performance, as some teams median scores improved when decomposed into sub-teams.

In time, the combination of human subject studies and agent-based modeling methodologies will deepen our understanding of cognitive style and its implications for both individual and team problem-solving behavior. In this initial foray, we found that some combinations of team structure and composition sacrifice robustness for median performance, and while others sacrifice median performance for robustness. This tradeoff is vital to team success and should be informed by the team’s overarching objectives.

Footnote

Acknowledgment

The first author’s research assistantship is supported by the Defense Advanced Research Projects Agency through cooperative agreement N66001-17-4064. Any opinions, findings, conclusions, or recommendations expressed in this paper are those of the authors and do not necessarily reflect the views of the sponsor.

Conflict of Interest

There are no conflicts of interest.

Data Availability Statement

The datasets generated and supporting the findings of this article are obtainable from the corresponding author upon reasonable request.

References

1.
Monalisa
,
M.
,
Daim
,
T.
,
Mirani
,
F.
,
Dash
,
P.
,
Khamis
,
R.
, and
Bhusari
,
V.
,
2015
, “
Managing Global Design Teams
,”
Res. Technol. Manag.
,
51
(
4
), pp.
48
59
.
2.
Paretti
,
M. C.
,
Layton
,
R. A.
,
Laguette
,
S.
, and
Speegle
,
G.
,
2011
, “
Managing and Mentoring Capstone Design Teams: Considerations and Practices for Faculty
,”
Int. J. Eng. Educ.
,
27
(
6
), pp.
1192
1205
.
3.
Soni
,
V. D.
,
2020
, “
Importance and Strategic Planning of Team Management
,”
Int. J. Innov. Res. Technol.
,
7
(
2
), pp.
47
50
.
4.
Kirton
,
M. J.
,
2004
,
Adaptation-Innovation: In the Context of Diversity and Change
,
Routledge
,
New York
.
5.
Messick
,
S.
,
1976
,
Individuality in Learning
,
Jossey-Bass Publishers
,
Hoboken, NJ
.
6.
Urban
,
J. M.
,
Weaver
,
J. L.
,
Bowers
,
C. A.
, and
Rhodenizer
,
L.
,
1996
, “
Effects of Workload and Structure on Team Processes and Performance: Implications for Complex Team Decision Making
,”
Hum. Factors J. Hum. Factors Ergon. Soc.
,
38
(
2
), pp.
300
310
.
7.
Kirton
,
M. J.
,
1977
,
Manual of the Kirton Adaption-Innovation Inventory
,
National Foundation for Educational Research
,
London
.
8.
Samuel
,
P.
, and
Jablokow
,
K. W.
,
2011
, “
Toward an Adaption-Innovation Strategy for Engineering Design
,”
Proceedings of the 18th International Conference on Engineering Design
,
Washington, DC
,
Aug. 28–31
, pp.
377
386
.
9.
Jablokow
,
K.
,
Teerlink
,
W.
,
Yilmaz
,
S.
,
Daly
,
S.
,
Silk
,
E.
, and
Wehr
,
C.
,
2016
, “
Ideation Variety in Mechanical Design: Examining the Effects of Cognitive Style and Design Heuristics
,”
Proceedings of the ASME Design Engineering Technical Conference
,
Charlotte, NC
,
Aug. 21
.
10.
Stum
,
J.
,
2009
, “
Kirton’s Adaption-Innovation Theory: Managing Cognitive Styles in Times of Diversity and Change
,”
Emerg. Leadersh. Journeys
,
2
(
1
), pp.
66
78
.
11.
Lapp
,
S.
,
Jablokow
,
K.
, and
McComb
,
C.
,
2019
, “
KABOOM: An Agent-Based Model for Simulating Cognitive Style in Team Problem Solving
,”
Des. Sci.
,
5
.
12.
Heininger
,
K.
,
Chen
,
H. E.
,
Jablokow
,
K.
, and
Miller
,
S. R.
,
2018
, “
How Engineering Design Students’ Creative Preferences and Cognitive Styles Impact Their Concept Generation and Screening
,”
Proceedings of the ASME Design Engineering Technical Conference
,
Quebec City, Canada
,
Aug. 26–29
.
13.
Radwan
,
N.
,
McComb
,
C.
,
Menold
,
J.
,
Jablokow
,
K. W.
, and
McTernan
,
J.
,
2022
, “
Towards Characterizing Cognitive Style Coping Behavior in Engineering Design
,”
Proceedings of the ASME Design Engineering Technical Conference.
,
St. Louis, MO
,
Aug. 14–17
.
14.
Buffington
,
K. W.
,
Jablokow
,
K. W.
, and
Martin
,
K. A.
,
2002
, “
Project Team Dynamics and Cognitive Style
,”
Eng. Manag. J.
,
14
(
3
), pp.
25
33
.
15.
Sonalkar
,
N.
,
Jablokow
,
K.
,
Edelman
,
J.
,
Mabogunje
,
A.
, and
Leifer
,
L.
,
2017
, “
Design Whodunit: The Relationship Between Individual Characteristics and Interaction Behaviors in Design Concept Generation
,”
Proceedings of the ASME Design Engineering Technical Conference, Vol. 7
,
Cleveland, OH
,
Aug. 6–9
.
16.
Hammerschmidt
,
P. K.
,
1996
, “
The Kirton Adaption Innovation Inventory Find Group Problem Solving Success Rates
,”
J. Creat. Behav.
,
30
(
1
), pp.
61
74
.
17.
Kurtzberg
,
T. R.
,
2010
, “
Feeling Creative, Being Creative: An Empirical Study of Diversity and Creativity in Teams
,”
Creat. Res. J.
,
17
(
1
), pp.
51
65
.
18.
Mannix
,
E.
, and
Neale
,
M. A.
,
2016
, “
What Differences Make a Difference?: The Promise and Reality of Diverse Teams in Organizations
,”
Psychol. Sci. Public Interest
,
6
(
2
), pp.
31
55
.
19.
Maier
,
T.
,
DeFranco
,
J.
, and
Mccomb
,
C.
,
2019
, “
An Analysis of Design Process and Performance in Distributed Data Science Teams
,”
Team Perform. Manag.
,
25
(
7–8
), pp.
419
439
.
20.
Chamakiotis
,
P.
,
Dekoninck
,
E. A.
, and
Panteli
,
N.
,
2013
, “
Factors Influencing Creativity in Virtual Design Teams: An Interplay Between Technology, Teams and Individuals
,”
Creat. Innov. Manag.
,
22
(
3
), pp.
265
279
.
21.
He
,
J.
,
Butler
,
B. S.
, and
King
,
W. R.
,
2007
, “
Team Cognition: Development and Evolution in Software Project Teams
,”
J. Manag. Inf. Syst.
,
24
(
2
), pp.
261
292
.
22.
Yost
,
C. A.
, and
Tucker
,
M. L.
,
2000
, “
Are Effective Teams More Emotionally Intelligent? Confirming the Importance of Effective Communication in Teams
,”
Delta Pi Epsil. J.
,
42
(
2
), p.
101
.
23.
Levitt
,
R. E.
,
2012
, “
The Virtual Design Team: Designing Project Organizations as Engineers Design Bridges
,”
J. Organ. Des.
,
1
(
2
), pp.
14
41
.
24.
Perišić
,
M. M.
,
Martinec
,
T.
,
Štorga
,
M.
, and
Kanduč
,
T.
,
2016
, “
Agent-Based Simulation Framework to Support Management of Teams Performing Development Activities
,”
DS 84: Proceedings of the DESIGN 2016 14th International Design Conference
,
Vienna, Austria
, pp.
1925
1936
.
25.
Herath
,
D.
,
Costello
,
J.
, and
Homberg
,
F.
,
2017
, “
Team Problem Solving and Motivation Under Disorganization—An Agent-Based Modeling Approach
,”
Team Perform. Manag. Int. J.
,
23
(
1–2
), pp.
46
65
.
26.
Crowder
,
R. M.
,
Robinson
,
M. A.
,
Hughes
,
H. P. N.
, and
Sim
,
Y. W.
,
2012
, “
The Development of an Agent-Based Modeling Framework for Simulating Engineering Team Work
,”
IEEE Trans. Syst. Man Cybern. Part A Syst. Humans
,
42
(
6
), pp.
1425
1439
.
27.
Xu
,
Y.
,
Liao
,
E.
,
Scerri
,
P.
,
Yu
,
B.
,
Lewis
,
M.
, and
Sycara
,
K.
,
2006
,
Coordination of Large-Scale Multiagent Systems
,
Springer
,
Boston, MA
, pp.
287
309
.
28.
Stolarski
,
V.
, and
Tilebein
,
M.
,
2009
, “
Diversity as a knowledge resource in top management teams–a framework for agent-based modeling
,”
42nd Hawaii International Conference on System Sciences
,
Big Island, HI
,
Jan. 5–8
, IEEE.
29.
McComb
,
C.
, and
Jablokow
,
K.
,
2022
, “
A Conceptual Framework for Multidisciplinary Design Research With Example Application to Agent-Based Modeling
,”
Des. Stud.
,
78
, p.
101074
.
30.
Hsu
,
S. C.
,
Weng
,
K. W.
,
Cui
,
Q.
, and
Rand
,
W.
,
2016
, “
Understanding the Complexity of Project Team Member Selection Through Agent-Based Modeling
,”
Int. J. Proj. Manag.
,
34
(
1
), pp.
82
93
.
31.
Martínez-Miranda
,
J
,
Aldea
,
A.
,
Bañares-Alcántara
,
R.
, and
Alvarado
,
M.
,
2006
, “
TEAKS: Simulation of Human Performance at Work to Support Team Configuration
,”
Proceedings of the International Joint Conference on Autonomous Agents and Multi-Agent Systems, 2006
,
Hakodate, Japan
,
May 8–12
, pp.
114
116
.
32.
Motamediyan
,
F.
,
Larsson
,
T.
,
Thompson
,
A.
, and
Motamediyan Dehkordi
,
F.
,
2012
, “
Impacts of Project-Overload on Innovation Inside Organizations: Agent-Based Modeling
,”
Int. J. Soc. Behav. Educ. Econ. Bus. Indus. Eng.
,
6
(
11
), pp.
2808
2813
.
33.
Kirkpatrick
,
S.
,
Gelatt
,
C. D.
, and
Vecchi
,
M. P.
,
1983
, “
Optimization by Simulated Annealing
,”
Science
,
220
(
4598
), pp.
671
680
.
34.
McComb
,
C.
,
Kotovsky
,
K.
, and
Cagan
,
J.
,
2015
, “
Lifting the Veil: Drawing Insights About Design Teams From a Cognitively-Inspired Computational Model
,”
Des. Stud.
,
40
, pp.
114
142
.
35.
Cagan
,
J.
, and
Kotovsky
,
K.
,
1997
, “
Simulated Annealing and the Generation of the Objective Function: A Model of Learning During Problem Solving
,”
Comput. Intell.
,
13
(
4
), pp.
534
581
.
36.
Yu
,
B. Y.
,
Honda
,
T.
,
Sharqawy
,
M.
, and
Yang
,
M.
,
2016
, “
Human Behavior and Domain Knowledge in Parameter Design of Complex Systems
,”
Des. Stud.
,
45
, pp.
242
267
.
37.
Lapp
,
S.
,
Jablokow
,
K.
, and
McComb
,
C.
,
2019
, “
Collaborating With Style: Using an Agent-Based Model to Simulate Cognitive Style Diversity in Problem Solving Teams
,”
Proceedings of the ASME Design Engineering Technical Conference
,
Anaheim, CA
,
Aug. 18–21
.
38.
Carnabuci
,
G.
, and
Diószegi
,
B.
,
2015
, “
Social Networks, Cognitive Style, and Innovative Performance: A Contingency Perspective
,”
Acad. Manag. J.
,
53
(
3
), pp.
881
905
.
39.
Blackburn
,
J.
,
Lapre
,
M. A.
, and
van Wassenhove
,
L. N.
,
2006
, “
Brooks' Law Revisited: Improving Software Productivity by Managing Complexity
,” Available at SSRN 922768, pp.
1
24
.
40.
Patrashkova-Volzdoska
,
R. R.
,
McComb
,
S. A.
,
Green
,
S. G.
, and
Compton
,
W.
,
2003
, “
Examining a Curvilinear Relationship Between Communication Frequency and Team Performance in Cross-Functional Teams
,”
IEEE Trans. Eng. Manage.
,
50
(
3
), pp.
262
269
.
41.
Patrashkova
,
R. R.
, and
McComb
,
S. A.
,
2004
, “
Exploring Why More Communication Is Not Better: Insights From a Computational Model of Cross-Functional Teams
,”
J. Eng. Technol. Manag.
,
21
(
1–2
), pp.
83
114
.
42.
Austin-Breneman
,
J.
,
Yu
,
B. Y.
, and
Yang
,
M. C.
,
2015
, “
Biased Information Passing Between Subsystems Over Time in Complex System Design
,”
ASME J. Mech. Des.
,
138
(
1
), p.
011101
.
43.
Jablokow
,
K. W.
,
2008
, “
Developing Problem Solving Leadership: A Cognitive Approach
,”
Int. J. Eng. Educ.
,
24
(
5
), pp.
936
954
.
44.
Stempfle
,
J.
, and
Badke-Schaub
,
P.
,
2002
, “
Thinking in Design Teams—An Analysis of Team Communication
,”
Des. Stud.
,
23
(
5
), pp.
473
496
.
45.
Demirel
,
S.
,
Macek
,
L.
,
Bruijnen
,
H.
,
Hakimi
,
M.
,
Böckler
,
D.
, and
Attigah
,
N.
,
2012
, “
Eversion Carotid Endarterectomy is Associated With Decreased Baroreceptor Sensitivity Compared to the Conventional Technique
,”
Eur. J. Vasc. Endovasc. Surg.
,
44
(
1
), pp.
1
8
.
46.
Nachar
,
N.
,
2008
, “
The Mann-Whitney U: A Test for Assessing Whether Two Independent Samples Come From the Same Distribution
,”
Tutor. Quant. Methods Psychol.
,
4
(
1
), pp.
13
20
.
47.
McHale
,
J.
, and
Flegg
,
D.
,
1986
, “
Innovators Rule OK- or Do They
,”
Train. Dev. J.
, pp.
10
13
.
48.
Silk
,
E. M.
,
Rechkemmer
,
A.
,
Daly
,
S. R.
,
Jablokow
,
K. W.
, and
McKilligan
,
S.
,
2021
, “
Problem Framing and Cognitive Style: Impacts on Design Ideation Perceptions
,”
Des. Stud.
,
74
, p.
101015
.