What Is the Relationship Between Psychological Safety and Team Productivity and Effectiveness During Concept Development? An Exploration in Engineering Design Education

What makes an engineering design team most effective? This elusive question is of utmost importance to organizations around the globe [1,2] due to the widespread belief that teams are more effective at generating solutions to complex problems than individuals alone. This increased team performance has been attributed to the range of knowledge and experience held by the team [3,4]. While engineering organizations around the world integrate teaming as a key aspect of their core business strategy [2,3], it is unclear what characteristics make a team productive.

To answer this question, Google’s People Operations division spent time trying to uncover what it was about teams in their organization that led some to succeed and others to falter [4]. In a project code-named “Project Aristotle,” the company explored whether the best teams were people with similar attributes, or if team success was more dependent on how often team members socialized or their intelligence. Surprisingly, the who part of the equation did not matter. High performance was not dependent on bringing together the most intelligent people. Some “good” teams had “smart” people who figured out how to break up the work evenly, while other “good” teams had “average” people who came up with ways to use each other’s strengths to their advantage [5]. Specifically, Google’s data indicated that psychological safety was critical to making the team successful.

Psychological safety, or “the shared belief that the team is safe for interpersonal risk taking” [6], has been found to be a consistent, generalizable, and multilevel predictor of outcomes in performance and learning across fields such as management, organizational behavior, social psychology, and healthcare management [7]. In addition, meta-analytic evidence identified a relationship between psychological safety, learning, and performance, showing that this relationship has the greatest impact on tasks that are complex, knowledge intensive, and involve creativity and sense making [8]. This is the very description of the skills needed in the engineering design process [9,10]. Particularly, facets of the psychological safety construct could relate to engineering design outputs, where feeling valued by one’s team, or feeling safe to take a risk and not fearing criticism for making mistakes [6], could drive the innovativeness and riskiness of the ideas that team members propose and select. Regardless, there is still limited evidence on the impact of psychological safety on engineering outputs.

While psychological safety has not been heavily explored in engineering, research in innovation management has provided evidence on why it may be an important area to explore. Specifically, research in this field has linked psychological safety to creativity by showing that it can help enable individuals to propose unique ideas and promote them to give constructive feedback to teammates [11,12], an important set of skills in engineering education [13]. These results indicate a possible relationship between psychological safety and team performance during the concept generation and screening stages of the engineering design process. Interestingly, the Comprehensive Assessment of Team Member Effectiveness (CATME), used widely in engineering education to create teams and assess team performance [14], contains themes of psychological safety [15,16]. However, research on this tool has only speculated about the role of psychological safety in undergraduate engineering team student projects [17–19]. Finally, while our own prior work has validated the longitudinal reliability of psychological safety in an engineering student sample [9], there have been limited investigations into the effectiveness or use of this measure on engineering team outputs.

In the light of the prior work, the goal of the current article is to explore the role of psychological safety on team productivity and effectiveness during the conceptual phases of the engineering design process. It is important to mention that while some studies look at the “innovation process” to promote team design outputs [20], our work focuses specifically on a psychological safety lens. The results of this study provide empirical evidence of the role of psychological safety in engineering design teams’ productivity. These results can be used to understand to what extent psychological safety shares a relationship with design outputs and to establish whether building upon existing interventions focused on psychological safety [21] may be worthwhile to pursue for fostering team psychological safety in engineering.

In its simplest form, the engineering design process consists of three phases: generation, evaluation (e.g., concept screening), and communication [22–24]. During concept generation, teams seek to develop creative ideas, or those that are both novel and useful [25]. Conversely, concept screening involves rating ideas in a go/no go fashion in an effort to evaluate new ideas quickly and prevent committing resources to potentially unsuccessful ideas [26].

In the midst of these stages, conflict can seep into the team atmosphere, where resistance to externally imposed task demands and interpersonal conflict can occur [22]. Specifically, when there are people of varying cognitive styles (i.e., an individual’s cognitive preference for solving problems [27,28]) and cognitive levels (i.e., an individual’s cognitive capacity to solve problems and display creative processes [28]), a cognitive gap between team members can occur [28,29]. While the cognitive gap can help diverse team members explore the solution space, it can also incite conflict if team members’ differences are not managed [28,29]. While some studies have shown that conflict holds value, such as engaging in problem-solving through argumentation [30,31], the prior work has also shown that such conflict is only beneficial if the psychological safety of the team is high, allowing members to tactfully challenge issues [6,11]. For example, low levels of psychological safety hindered the performance of employees in manufacturing companies, causing individuals to feel a “lack of growth” and “not be heard” as they struggle to improve the product [32]. In addition, research in hierarchies of hospital workers communicating through intense, unpredictable contexts [11], as well as cardiac departments trying to learn new technologies [33], has shown that when team psychological safety is high, members are more prone to speak out against problems and dismiss fears of being criticized for making mistakes [6,34]. This safety is built upon emotional interactions and deep conversations within a team that convey to team members how individuals want to portray themselves and how others make them feel [5]. To understand these interactions better, analysis of audio recordings can help to break down verbal communication into meaningful bits of information [35], as understanding teams’ trends in psychological safety can be difficult without this context.

While outside the context of engineering, research has also linked psychological safety to employees’ feelings of vitalities and ultimately their involvement in creative work [36]. This is critical to consider, as techniques such as the Consensual Assessment Technique (CAT) [37] have been used in engineering design to rate the creativity [38–40], a critical design criterion in engineering design [41]. The recent work has also looked at examining ideas as incremental to radical changes to the existing solution [42,43], which show promise as a technique for examining design outputs [44]. Although an investigation of creative outputs and/or radical ideas is not within the scope of this article, investigating the broader outputs helps to establish the direction of how teams may perform. Specifically, prior studies show a connection between psychological safety and creativity such that individuals are more likely to propose unique ideas and engage in the process of giving constructive feedback within the team [11,12]. Similarly, another study showed that healthcare teams with low psychological safety tended to avoid sharing creative ideas [45], whereas the prior work in manufacturing showed that psychological safety contributed to divergent thinking, creativity, and risk-taking [46]. However, the ability to produce creative outputs does not come without a cost, as the team’s ability to explore the solution space can come from the aforementioned cognitive gap, which could also be detrimental to team performance if interpersonal issues due to the gap are not resolved [28,29]. In addition to this gap, the design task can also impact how team members produce design outputs [47], as well as background knowledge on the task itself [48], either limiting or promoting performance from specific individuals.

In addition to this empirical work, reviews of the psychological safety literature have identified several promising areas for research, including adopting a dynamic view of psychological safety to understand how the construct is established, builds, wanes, and/or disappears completely over time [7,8]. This is important in the context of engineering because a lack of psychological safety in a team environment may manifest itself differently throughout the design process [9]. For example, the prior work in healthcare showed that teams with low psychological safety refrained from sharing novel ideas with each other [45]. This finding suggests a potential relationship between psychological safety and concept generation in the engineering design process. Establishing whether this relationship exists is important because researchers have linked freedom to express creative ideas to the number of ideas, or the fluency of ideas [49,50]. In some cases, high psychological safety may stimulate the production of new products and services through feeling interpersonally safe to share their ideas [51]. In addition, speaking up and embracing mistakes may encourage people to suggest unique ideas through, effectively increasing creativity and innovation in teams [11,12,34]. However, while feeling interpersonally safe to generate novel ideas may help overcome the fear of risk-taking [6,12,34], it does not necessarily guarantee that team members can overcome barriers to brainstorming in groups such as “production blocking,” where only one person at a time can speak [52]. This is echoed in prior findings where nominal groups (individuals working by themselves) tended to generate more ideas and more original ideas than their interactive group counterparts [53,54]. Conversely, “social loafing” may occur in groups when individuals do not feel as accountable in the group for evaluation purposes (such as a project grade) in comparison to an individual evaluation [55]. Therefore, these types of brainstorming issues may hinder performance if they happen to override high team psychological safety. Another aspect to consider at the concept generation stage is that while some literature supports the benefits of generating many ideas in terms of originality [56,57] and allowing teams to explore a diverse pool of ideas [58], other literature has found that larger quantities of ideas do not necessarily mean that those ideas will be high quality and sometimes the opposite [59,60], which should be considered when making any claims about psychological safety and ideation. In other words, idea production (more ideas) does not necessarily equal idea effectiveness (producing the right ideas).

Psychological safety may also play an important role in the concept screening stage of the engineering design process. In fact, it is thought that high psychological safety is correlated with a high level of agreeableness among team members [61], which may impact the types of ideas team members screen out during the design process. For example, low levels of psychological safety may impact individuals to be biased toward selecting their own ideas, an effect known as ownership bias [62,63]. This could impact concept screening due to the relationship among psychological safety, trust (not the same as psychological safety, but can serve as an input [7]), and openness of communication [64], especially when it comes to errors and concerns [65]. In particular, ownership bias can deteriorate the sense of importance in collaboration through enticing individuals to choose their ideas over others’ ideas [66], potentially impacting selection processes that can impact the final design [67]. This is problematic because the lack of collaboration goes against the requirements for high psychological safety [6]. Conversely, the halo effect has been expressed by team members in an engineering design context, where they select their team members’ ideas over their own during concept screening to express the “goodness” of an idea [68], as based on a notion of overall quality from [69]. This is because the idea rater perceives that other members produce higher quality designs in comparison to their own designs for the design task [68]. While the prior work has demonstrated the effects of ownership bias [66], recent work on an engineering sample identified that ownership bias may only be present when taking into account the “goodness,” a measure of design quality, of the idea [70]. Thus, the relationship between psychological safety and ownership may be mediated by such quality measurements. However, how interactions outside the classroom occur could confound with the in-class building and waning of psychological safety that may contribute to design outputs, as students may use technology to communicate (e.g., texting/instant messaging applications and social networking systems) to work on assignments and/or study together [71].

While findings from these aforementioned studies provide the foundation for why psychological safety may impact engineering design outputs, there has been limited evidence on its role in the productivity and effectiveness of concept development tasks such as those present in engineering design. The current study was developed to fill this void.

The main goal of the current article was to explore the role of psychological safety on engineering team performance in the conceptual phases of the design process. Specifically, the following research questions were explored:

RQ1: What is the relationship between psychological safety and the fluency and goodness of the ideas that teams develop during concept generation? Our hypothesis was that as psychological safety increases, the total number of ideas (fluency) created per team would increase, as would the average idea goodness rating per team. This is important during concept generation, as a greater number of ideas per team could present a diverse pool of designs to choose from [58], allowing teams to explore the solution space. Specifically, psychological safety has been shown to facilitate the contribution of ideas [7] and encourage people to take initiative to develop new products and services [51]. Furthermore, because idea goodness is judged by team members, it may be a way of showing that a team member has more trust from the perspective of team members generating viable ideas, which can influence the psychological safety of teams positively [65] as well as promote agreeableness within the team [61].

RQ2: What is the relationship between psychological safety and team performance during concept screening? Our hypothesis was that as psychological safety decreases, the incidence of ownership bias at the team level would increase. Particularly, ownership bias is linked to performance through representing teams’ lack of collaboration via members within teams that overlook others’ potentially successful ideas [66]. Because ownership bias is most noticeable when team members are given the option to either select their ideas or others’ ideas, we decided to investigate this phenomenon during concept screening. Furthermore, we proposed that a decrease in perceptions of psychological safety at the individual level would also cause ownership bias to increase among individuals. This is because ownership bias is related to team members having a preference for their own ideas [68,70], causing them to lose sight of the importance of collaboration. In relation to idea goodness, an increased selection of one’s own ideas that are rated low by others can be construed as a sign that ownership bias is existent [70].

To answer the research questions presented earlier, we conducted an empirical study at a large northeastern university over the first project of a cornerstone engineering design course from semester during Summer 2018 to Spring 2020. Figure 1 depicts the study timeline. These time points were chosen because they represent milestones in the engineering design process for a team [22], and we can extract performance outputs as a result of team interaction for the analysis. Further details of the study design are presented in the remainder of this section.

Sixty-nine engineering design student teams, comprising 263 participants (188 men and 75 women), participated in this study. All participants were enrolled in a first-year engineering design course at a large northeastern university. This study was integrated into the curriculum, and the students were graded based on their participation.

The study was completed over the course of 2 years with a first-year cornerstone engineering design class. Specifically, 11 sections of this course were studied in the current investigation; seven of which took part over the course of a typical semester (15 weeks), while four transpired over a condensed summer session (6 weeks; see Table 1 for the summary). The same course schedule was followed and adhered to, and the psychological safety of the teams was analyzed over the same five time points in all instantiations of the course (see Table 1). Each design session at their respective time point lasted approximately 1 h and 50 min in every semester, making the time to complete each activity roughly equal in length. At the end of each time point, students completed an electronically delivered seven-question psychological safety survey developed by Edmondson [6], shown in the Supplemental Material on the ASME Digital Collection. These survey questions center around the degree to which team members feel comfortable making mistakes without criticism, bringing up difficult issues intended to help the group, and feeling accepted and valued as a team member [6]; all of which are important for providing feedback in an engineering team [13]. A popular example of one of these questions is, “If you make a mistake on this team, it is often held against you” [6]. Importantly, this investigation focused mainly on concept generation and screening (time points 2 and 3), however, it is important to state the previous stages that lead into generation and screening practices, and what outputs come out of these stages that feed into later design stages. All participants consented at the beginning of the study based on the institutional review board guidelines established at the university. The remainder of this section highlights what happened for each section of this course at each time point with respect to the current study.

At time point 1, 3- and 4-person teams were formed using the 32-item Kirton's Adaption-Innovation (KAI) inventory (validated across the general population and engineers) to determine their cognitive styles [27,28]. Specifically, although not discussed in the current study, half of the teams were constructed to be homogeneous (all KAI scores within a ten-point range), while the other half was constructed to be heterogeneous by team KAI score. Next, students were presented with a design challenge, which differed by term/instructor of the course (see Table 1 for descriptions). The teams then conducted in-depth context research on their design problem, which served as their area of focus for their design project. At the end of the class, the students completed the first psychological safety survey.

During time point 2, students attended a lecture on customer needs and developed their problem statements. After this, an innovation module that focused on the importance of creativity in engineering design was then completed. Next, the participants were guided through a series of idea generation exercises where they were asked to individually sketch as many ideas as possible in a 15 min session in nominal groups. At the end of this period, the instructor collected the ideas that were scanned for the analysis. After this, participants completed the second psychological safety survey.

During time point 3, participants were led through a concept selection activity where they individually assessed all of the ideas generated by their design team. Specifically, students were provided the ideas their team generated in time point 2 in a random order and asked to individually assess all of the ideas generated by their design team by categorizing the ideas using a concept screening sheet into “Consider” or “Do Not Consider” categories (see Fig. 2 for an example of the concept scoring sheet). Ideas in the “Consider” category were concepts that the participant felt would most likely satisfy the needs for the problem statement for the course project, while ideas in the “Do Not Consider” category were concepts that the participants felt were not adequate in satisfying the design goals. This was continued until all ideas from the group were assessed. The students then discussed the ideas they screened and formed two piles as a group—“Consider” and “Do Not Consider.” They were tasked with picking out four distinct ideas to prototype in the next design session. At the end of this time point, the third psychological safety survey was completed.

At time point 4, students were tasked with developing low-fidelity prototypes of the idea they selected during time point 3 using commonly available materials (e.g., foam core, cardstock, post-its). From there, students were given a few minutes to develop their “elevator pitch” to promote their prototype. Then, the students divided off into eight new teams for 15 min to share their elevator pitch and receive feedback on their idea. At the end of this session, all participating students completed the fourth psychological safety survey.

The project ended at time point 5, in which the final deliverables were completed including a formal PowerPoint presentation, a final design report, and a high-fidelity prototype including a CAD rendering of the design. After all groups presented their presentations, students were completed the fifth and final psychological safety survey.

To answer our research questions, several metrics were utilized including: idea fluency, idea goodness, ownership bias, and psychological safety. Each metric is defined in detail in the remainder of this section.

Ownership Bias: Ownership bias describes a participant’s preference or bias for their own ideas during the design process [70]. To measure ownership bias, the continuous parameter idea goodness was applied to six distinct metrics to analyze the continuous parameter of the percentage of ideas selected by the idea generator themselves, or by other team members on both a high level (not considering the idea goodness, but purely the percentage selected), or finer level (ideas designated as “low” or “good” by the team members who did not create the idea). Thus, several metrics were developed and calculated as follows:

During the study, three teams were removed from the Spring 2020 semester due to the teams being broken up part of the way through the semester, invalidating their results. For the analysis at time point 2, two of the teams were removed due to issues with team members either not turning in all ideas or only one person responding to the psychological safety survey. This left 67 engineering design teams that generated an average of 6.58 (SD = 2.11) ideas per person, where the average psychological safety score was 6.03 (SD = 0.505). Furthermore, due to issues with team members not evaluating all ideas from their other teammates, two teams were removed from the idea goodness and ownership bias analyses at time point 3. This left 67 teams that selected an average of 68.30% (SD = 11.03) of the ideas generated by their respective team members, where the average psychological safety score was 6.11 (SD = 0.497). To ensure that the formation of teams via cognitive style did not confound the results presented in the research questions, a hierarchical regression was conducted. We conducted two hierarchical regressions with two steps each. The first hierarchical regression predicted psychological safety at time point 2, where KAI (homogeneous or heterogeneous team) was entered as a control variable in the first step, and idea fluency was entered in the subsequent step. The second hierarchical regression predicted psychological safety at time point 3, where KAI was entered as a control variable in the first step, and idea goodness and the six cases for ownership bias were entered in the subsequent step. Before conducting a hierarchical multiple regression, the relevant assumptions of this statistical analysis were tested. The assumption of singularity was also met as the independent variables (team idea fluency, mean team idea goodness, and each of the six metrics used in ownership bias) were not a combination of other independent variables. From there, an examination of the Mahalanobis distance scores indicated no multivariate outliers. Residual and scatterplots indicated the assumptions of normality, linearity, and homoscedasticity were all satisfied. In the case of time point 2, controlling for KAI was found to be statistically insignificant, with adjusted R² = –0.013, and F(1, 66) = .120, p = 0.730. In the case of time point 3, controlling for KAI was found to be statistically insignificant, with adjusted R² = –0.015, and F(1, 63) = 0.077, p = 0.782. The remainder of this section presents the results in reference to our research questions. The statistical data were analyzed via the SPSS v.26. A value of p < 0.05 was used to define statistical significance [72].

RQ1: What is the relationship between psychological safety and the fluency and goodness of ideas teams develop during concept generation?

The goal of our first research question was to identify if a relationship existed between psychological safety and engineering team outputs during the concept generation process. Specifically, we hypothesized that as team psychological safety increased, the total normalized number of ideas (fluency) per team would increase because the prior work conducted outside of engineering has shown that psychological safety facilitates the contribution of ideas [7] and new products and services [51]. Furthermore, because idea goodness may tap into feelings of trust within the team and influence an increase in psychological safety [65] and also increase agreeableness within the team [61], we also hypothesized that as psychological safety increases, the average idea goodness would also increase.

Before the analysis, the validity of team aggregations of psychological safety at time point 2 was verified because psychological safety is a team-level construct. This was achieved through interrater agreement calculations. The results revealed an acceptable level of agreement, and thus, the construct was considered valid at this time point (r_wg = 0.88, ICC(1) = 0.178, ICC(2) = 0.431) [73]. This is based on the criteria defined in LeBreton and Senter [73], where our ICC(1) estimates are medium effects (around ICC(1) = 0.10 is considered as such) and r_wg values indicate strong agreement (r_wg between 0.71 and 0.90 is considered as such). In addition, statistical assumptions were checked before the analysis. Specifically, requirements for homoscedasticity were met as assessed by visual inspection of a plot of standardized residuals versus standardized predicted values. In addition, normality was confirmed by visually inspecting the histograms and Q-Q plots.

Once assumptions were validated, two linear regression analyses were conducted. The first linear regression used the independent variable of psychological safety during time point 2, concept generation, and the dependent variable idea fluency. One outlier (15 ideas per participant with a psychological safety score of 6.95) was present in the data, which were transformed into the next highest value (11.67 ideas per participant), as leaving the outlier would result in a different statistical conclusion. The results of the regression analysis significantly predicted a relationship between these variables, F(1, 65) = 5.752, p = 0.019. Specifically, psychological safety accounted for 6.8% of the explained variance in idea fluency, a small effect size according to Cohen [74]. The regression equation was expressed as follows: predicted normalized idea fluency = 12.98–1.073x (psychological safety). A scatterplot of this is shown in Fig. 3. This finding did not support our hypothesis; psychological safety was shown to facilitate the contribution of ideas in a negative manner, the opposite of what prior research pointed toward [7]. Although these results were surprising at first, we analyzed the data even further to understand why psychological safety was inversely related with normalized idea fluency. Specifically, a multivariate linear regression analysis was conducted with the dependent variables being the standard deviation of team idea fluency, the maximum idea fluency within a team, the minimum idea fluency within a team, the minimum idea goodness within a team (where the idea goodness scores were averaged for each participant to indicate the average quality of a person’s ideas), and the standard deviation of team idea goodness. The independent variable was psychological safety at time point 2.

Our results from the multivariate linear regression revealed that while the standard deviation of idea fluency (R² = –0.015, p = 0.882), maximum idea fluency (R² = 0.010, p = 0.199), minimum idea goodness within a team (R² = 0.039, p = 0.06), and the standard deviation of team idea goodness (R² = 0.005, p = 0.248) were not significantly related to psychological safety at time point 2, minimum idea fluency was significantly related. Further analysis via a regression analysis significantly predicted a relationship between these variables, F(1, 65) = 4.596, p = 0.017. Specifically, psychological safety accounted for 7.0% of the explained variance in idea fluency, a small effect size according to Cohen [74]. The regression equation was: predicted minimum idea fluency = 10.075–0.888x (psychological safety). A scatterplot of this is shown in Fig. 4. This result helped to explain the inverse relationship between psychological safety and normalized idea fluency in the main analysis, as teams tended to have at least one team member exhibit signs of social loafing, even when the team’s psychological safety increased in comparison to teams with low psychological safety (usually around a score of 4 for this study). The individual may not have felt the need to contribute as much, lowering average team motivation [75]. This can be seen as an unintended effect of high psychological safety, similar to what has been found in the previous literature [75,76]. To explain the findings of our results even further, we performed a linear regression that used the independent variable normalized idea fluency and the dependent variable mean team idea goodness. While the results of the regression analysis failed to significantly predict a relationship between these variables, F(1, 64) = 3.770, p = 0.057, we were able to see a trend beginning to form based on the equation: predicted idea goodness = 0.393–0.048x (normalized idea fluency). Regardless, the results did not meet criteria for statistical significance, indicating no relationship. However, because the prior work found that idea quantity does not necessarily equal quality [59,60], the future work may benefit from a larger sample size.

The second linear regression had the independent variable of psychological safety during time point 3 and the dependent variable idea goodness. The results of the regression analysis identified that psychological safety significantly predicted idea goodness, F(1, 65) = 4.937, p < 0.05. Specifically, psychological safety accounted for 7.1% of the explained variance in idea goodness. The regression equation was expressed as follows: predicted idea goodness = 0.32 + 0.059x (psychological safety). A scatterplot of this is shown in Fig. 5. This finding supported our hypothesis such that team psychological safety would promote higher levels of team idea goodness based on the notion that higher psychological safety is associated with agreeableness among team members [61].

Although psychological safety was found to be associated with the total number of ideas generated per team in a negative manner, it was associated with more viable ideas. This result indicates that as psychological safety increased, so did the average idea goodness of the team. Because psychological safety impacts the team’s likelihood to take risks [6], rating others’ ideas highly could be a way of “risk-taking.” This implies that team members are comfortable enough that they are willing to try more of their team members’ ideas. This can also be alluded to trust being an important factor in psychological safety [34], where high trust in team members’ abilities can promote risk-taking in the form of selecting others’ ideas.

RQ2: What is the relationship between psychological safety and team performance during concept screening?

The goal of our second research question was to examine if a relationship existed between team psychological safety and performance outputs from concept screening. Specifically, we hypothesized that as team psychological safety decreased, the incidence of ownership bias would increase. This is based on using ownership bias as a proxy to measure the lack of sense of the importance of collaboration, where team members select their own ideas without considering others’ ideas [66]. This is particularly critical for subsequent design outputs, as selection processes can impact outputs such as the final design [67]. Furthermore, we hypothesized that as perceptions of psychological safety decreased at the individual level, incidence of ownership bias would increase [58].

Similar to in RQ1, before the analysis, the validity of team aggregations of psychological safety at time point 3 was verified through interrater agreement calculations (r_wg = 0.87, ICC(1) = 0.129, ICC(2) = 0.354) [73]. In addition, statistical assumptions were checked. Specifically, requirements for homoscedasticity were met, as assessed by visual inspection of a plot of standardized residuals versus standardized predicted values. Furthermore, normality was confirmed by visually inspecting the histograms and Q-Q plots.

Once assumptions were verified, six linear regression analyses were conducted at the team level as well as an investigation of individual perceptions of psychological safety for all six cases using a multilevel analysis [77]. Four of the six cases are described here, where the remaining two analyses were variations of the percentage of team members’ ideas selected, similar to that of the idea goodness cutoffs used in percentage of own ideas selected. It is important to state that teams where individuals did not evaluate their own ideas were removed from analyses for selecting their own ideas, leaving 65 teams to be analyzed. In the case where not all team members evaluated all others’ ideas, those teams were removed for analyses focused on individuals selecting others’ ideas.

The first linear regression used the independent variable of psychological safety during time point 3, concept screening and the dependent variable percentage of own ideas selected. The results failed to show a statistically significant relationship between psychological safety and percentage of own ideas selected, F(1, 63) = 0.813, p = 0.371. To see if this ownership bias was contingent on the quality of the ideas, a second linear regression analysis used the dependent variable named percentage own ideas with goodness score below 0.5. However, the results failed to reveal a statistically significant correlation, F(1, 63) = 0.982, p = 0.325. Finally, a third linear regression analysis was conducted with percentage own ideas with goodness score above 0.5. However, the results again revealed no statistically significant relationship, F(1, 63) = 0.032, p = 0.858. Furthermore, none of the results revealed any statistical significance for ideas being selected by other team members.

These results refute our hypothesis in the sense that ownership bias is not associated with lower team psychological safety nor perceptions of psychological safety due to the lack of statistical significance. Furthermore, the halo effect [68] is not even present, where team members tend to select others’ ideas over their own. These results convey that while psychological safety shared a positive relationship with team perceptions of idea goodness in RQ1, RQ2’s findings suggest that idea quality perceptions for selecting ideas others’ are unrelated to psychological safety.

The main goal of this study was to explore the role of psychological safety on engineering team productivity in the conceptual phases of the design process. The main findings of this study are as follows:

• Psychological safety was significantly negatively correlated with the number of ideas (fluency) produced by a team

• Psychological safety was significantly positively related to team idea goodness.

• Ownership bias failed to share a relationship with both the team level and individual perceptions of psychological safety.

This result indicates that while psychological safety may not increase team productivity in terms of the number of ideas produced, it may impact team effectiveness in coming up with viable candidate ideas to move forward in the design process. The finding that psychological safety was significantly and negatively related to the total number of ideas generated (fluency) by the team alludes to literature that emphasizes the dark side of psychological safety in terms of social loafing [75] and unethical behaviors [76]. Furthermore, while prior research indicates that members who generate ideas in a team tend to offer more ideas than individuals working independently [54], this was not the case in our study. Although feeling interpersonally safe to generate novel ideas may overcome the evaluation apprehension or fear of being judged and looking unintelligent [6,12,34], other factors may bear more weight in this process, such as other barriers to brainstorming, such as production blocking and social loafing [52,55]. This can be seen where production blocking allows only one person to speak at a time [52], and individuals do not hold themselves accountable as a result of social loafing [55], which may be explained by the minimum idea fluency from one of the individuals decreasing as psychological safety increases. However, the ideas generated during the concept generation stage tended to be of high subjective quality based on the idea goodness ratings [68]. Because idea goodness can be facilitated by feelings of trust within the team, this may have influenced psychological safety in a positive manner [65], and thus, idea goodness increased.

In contrast to concept generation, psychological safety influenced the concept screening stage in a positive manner at the level of idea goodness. When team members feel that it is safe to take risks, they may be more likely to accept others for being different, value each other’s skills, and offer honest, negative feedback about the quality of the generated ideas without team members feeling as if they have been rejected or their efforts undermined. When team members feel that it is safe to take risks, they may be more likely to accept others for being different, value each other’s skills, and offer honest, negative feedback about the quality of the generated ideas without team members feeling as if they have been rejected or their efforts undermined [6]. When ideas can be critically vetted without threatening the egos of teammates [11], better solutions result from the perspective of idea goodness [68], as demonstrated in the results of these studies. In the creative process of engineering design [41], concept screening is where the benefits of psychological safety are salient. This is also apparent from the perspective of ownership bias, where critical signs of bias would have been apparent in the dependent variable of the average percentage of own ideas with low goodness selected [68,70]. However, there was no statistically significant correlation, exhibiting that ownership bias and psychological safety were not strongly related.

One conclusion of our results is that psychological safety exerts differential effects on creative processes. For example, psychological safety was found to be significantly and negatively associated with idea fluency during concept generation, whereas idea goodness was positively related during concept screening. However, psychological safety was found to not be impactful for the percentage of own ideas selected nor team members’ ideas selected during the examination of ownership bias during concept screening, meaning that increases in psychological safety do not impact the percentage of ideas that the individual chooses of their own or the percentage that the team chooses of others’ ideas.

While this study presents some interesting results to further broaden our view of how psychological safety plays a role in engineering design student project trajectories, such results do not come without limitations. First, many factors can influence the number of ideas that an individual proposes during concept generation; these might include their amount of tacit knowledge about the design problem or a tendency to shyness, among others. Specifically, both design task [47] and prior background knowledge [48] can impact design outputs, potentially impacting how many ideas that an individual feels that they can generate (idea fluency), or how well they understand the task to feel safe to agree with others’ ideas (idea goodness). In addition, according to the previous work related to creativity [25], other individual qualities can influence or inhibit their creativity. Because individual characteristics may influence idea fluency during concept generation, it may be difficult to determine the impact of psychological safety on concept generation, thus limiting our results. Specifically, some students may produce ideas at a slower rate than other students, where the limited time to produce ideas may have placed unintended bounds on teams’ idea fluency. Furthermore, the lack of team interaction at such an early stage in the design process may contribute to the outcome of a weak correlation, as psychological safety requires a significant amount of interaction and takes time to manifest [7].

Furthermore, the combination of idea goodness and team psychological safety does not tell the full story behind interactions between specific individuals during concept screening, as psychological safety is a team construct [6]. For example, if one member does not get along with one other individual and purposely does not consider their ideas, this would unfairly decrease the idea goodness of that individual’s ideas, despite this team having relatively high psychological safety. However, this can be analyzed through an ownership bias lens, where an idea generated by the original idea generator is selected despite having a lower idea goodness based on the ratings of others in the same team [70]. That being said, the idea goodness ratings in this study were simplified in comparison to an earlier study [70], which may be why very good ideas were not rated as highly and very poor ideas were not rated as negatively. Because these analyses rely on definitions from Toh et al. [68] to separate the “poor” ideas from “good” ideas while using a “majority rules” method, binning the results in such a way removes some of the details of the degree of goodness. Furthermore, as psychological safety is a team construct and is aggregated to the team level [6], the ownership bias calculations were aggregated to the team level as well. This makes it difficult to detect whether ownership bias is occurring in just one or two individuals or the team as a whole. In addition, results that show that any incidences of ownership bias could be due to some other factors beyond psychological safety, such as gender [68], which were not explored in this study.

In addition to limitations presented in concept screening analyses, full interpretation of the idea goodness scores is limited until more qualitative data are gathered from team members’ reasonings for how they decide to select others’ ideas. While higher psychological safety can increase a team’s likelihood for risk-taking, it is also known to impact an individual’s ability to speak up in a group when they believe there is an issue [6]. Therefore, rating ideas poorly could be a way of “speaking up.” The willingness to speak up is critical, as the success of a final design is largely dependent on the concept generation and concept selection stages of a project [10]. In other words, if poor ideas are not detected and removed in the early stages of a design, the end result could be catastrophic. Conversely, the inability to manage cognitive gaps may prevent team members from considering others’ ideas, preventing the team from exploring the solution space [27,28]. While results favor the risk-taking aspect, further analysis is needed to ensure this. Importantly, the ratings from this study do not take the originality of ideas into account, which can further confound results. Furthermore, while idea goodness through nonexpert ratings has been validated in other studies [68,70], we understand that individual perceptions of an idea’s quality can be subjective. To take a more objective approach, quasi-expert ratings of ideas from incremental to radical can be used in conjunction with KAI [42,43] and team psychological safety to explain how individuals’ feelings about their team and their inherent traits can impact design outputs.

In addition to specific limitations in concept generation and concept screening, the causal direction of psychological safety should be discussed as well. Because the psychological safety survey is taken at the end of class right after the activity, we assume that the psychological safety scores would not have been impacted much, if at all, throughout the duration of each activity. This is based on the notion that psychological safety takes time to manifest [7]; therefore, not much of a change is expected before and after each activity at one of the time points. Furthermore, the building and waning of psychological safety could take place outside of the classroom due to other forms of communication outside of class time when working on assignments and/or studying together [71], making the activity itself less likely to cause the team psychological safety to change. As this study is one of the first to examine psychological safety through multiple time points, and while we do not know the causal direction, it is beneficial to understand how psychological safety impacts team engineering design outputs. This can be critical for establishing future intervention methods aimed to improve psychological safety, as understanding whether there is any relationship between psychological safety and design outputs substantiates the potential benefit of focusing on increasing psychological safety to improve team performance. While the recent work has focused on these initiatives through role-based interventions [21], more work is needed to understand how such interventions can feed into engineering design team performance.

Along with the lack of causal direction of the activities, it is also important to discuss potential confounding effects of how KAI may impact concept generation and concept screening outputs. While these outputs may have an impact, our preliminary analyses have shown that KAI shows no statistical significance at all time points, leading us to believe that KAI is not impactful on productivity outputs. Therefore, an investigation of the potential impact of cognitive style (via KAI) on psychological safety would be more suitable for rating ideas from incremental to radical [42,43].

Although the current study sheds some light on how psychological safety impacts the activities of students during concept generation and concept screening, further investigation must be done to determine what types of verbal interactions impact the building or waning of psychological safety in engineering design teams along the way. Based on reviewing the team psychological safety scores at each time point, no particular trend could be depicted as most teams’ scores fluctuated throughout the trajectory of the design project, suggesting that some underlying factors could point to drops in psychological safety at various time points for teams. Similar to the trends exhibited in the study by Miller et al. [9], which specifically looked at the evolution of psychological safety over the time-steps, some teams started out with a high team psychological safety score and increased throughout the course of the project as the team members grew closer with each other, whereas some teams experienced a dip in team psychological safety at time point 2 (concept generation), time point 3 (concept screening), or time point 5 (final deliverables deadline). In general, psychological safety scores tended to be on the high end (close to 7), and while the cause for this sample remains unclear, the work outside of engineering education showed that external factors, such as inherent cultural and learning behaviors [8], can contribute to either high or low trends in psychological safety for specific groups. Furthermore, although design outputs during concept generation and concept screening have been gathered, outputs of the end product at time point 5 can be examined in a future study to develop an expanded view of how team psychological safety impacts the final product from each team.

In addition to teams’ psychological safety at individual points, a positive skew in psychological safety appeared for most teams, and team-level aggregate scores may have obscured individual members who reported low psychological safety, which is a point team scholars have highlighted [78]. Although individual perceptions of psychological safety were statistically insignificant in most incidences of analyzing engineering design outputs, further analysis is needed to uncover why some members had lower perceptions of psychological safety compared to others. These points suggest that a qualitative analysis of audio recordings [35] during these time points is important in determining how the interactions impact students’ abilities to perform optimally relative to their abilities during concept generation and concept screening. Finally, these results focused on a student group of designers, which may produce different design outputs from industry professionals. Therefore, the generalizability of these results is limited to design groups in education.

Special thanks are given to Janice Gong in analyzing the reliability of the psychological safety scale, Katie Heininger in collecting data for Summer 2018, and Randall Doles in assisting with the coding of the matlab codes used to quantify the raw data. We also would like to thank our participants for their help in this project.

• National Science Foundation (Grant No. 1825830).

There are no conflicts of interest.

The datasets generated and supporting the findings of this article are obtainable from the corresponding author upon reasonable request.