## Abstract

In undergraduate engineering programs, recent emphasis has been placed on a more holistic, interdisciplinary approach to engineering education. Some programs now teach product design within the context of the market, extending the curriculum to topics outside of scientific labs and computational analysis. However, it is unknown to what extent engineering students already understand the systems and contextual factors associated with product design, and also what characteristics or experiences have led students to these ways of thinking. This study analyzes survey and concept map data collected from 154 students in a third-year engineering design course. The aim is to understand how student backgrounds and experiences influence their mental models of product design. Data were gathered from surveys on student backgrounds and experiences, along with concept maps that were generated by the students at the beginning of a product design course. The concept maps were analyzed in a quantitative manner for structural and thematic elements. The findings show that several background attributes influence student conceptions of product design. Academic major appeared to have the largest impact on a variety of variables. Additionally, prior work experience, enrollment in a master’s program, and the presence of an engineering role model at home all showed significant impacts on design conceptions. By analyzing and understanding how the unique backgrounds of students lead to differences in thought, educators can adjust their curricula to more effectively teach design concepts to students of various backgrounds and experiences.

## 1 Introduction

Traditional undergraduate engineering programs emphasize technical knowledge, with courses in mathematics, physics, mechanics, thermodynamics, and other quantitative topics. While these subjects are undoubtedly critical for aspiring engineers, they can often overpower the importance of design education that also includes non-technical factors such as the markets in which designed artifacts must thrive [1]. This has prompted many institutions to reevaluate their design education curricula, making room for a more holistic approach to engineering design that emphasizes both technical skills and business acumen. Examples of design-related engineering education initiatives include the conceive, design, implement, and operate approach [2]; integrative science, technology, engineering, and mathematics (STEM) education [3]; the proliferation of capstone design courses [4], and the rise in project-based learning (PrBL) [5]. Many of today’s engineering students are now receiving some level of training in interdisciplinary design topics that correspond to the business needs of today’s dynamic professional environments [6]. Given the interdisciplinary nature of product design, it is particularly interesting to consider how diversity within the student population can influence the approach of this topic.

It is widely accepted that students’ individual backgrounds and experiences influence their initial knowledge and conceptions surrounding a topic prior to beginning coursework [711]. This study focuses specifically on how engineering student backgrounds influence the breadth and depth within their conceptions of product design prior to beginning a course on the topic. The primary research question is: How do the backgrounds and academic profiles of third-year engineering students—specifically, their academic major and whether they have meaningful work experience, grew up with an engineering role model, and intend to pursue a master’s degree—influence their conceptions of product design, as measured through the breadth and thematic contents of individually generated concept maps?

At the beginning of a third-year undergraduate engineering design course, data were collected from 154 students through a survey and a concept mapping activity. The survey gathered details about the students’ backgrounds and academic profiles, and the concept maps were generated individually around the central concept of “product design.” These maps were explored in a quantitative manner, analyzing both the structural and thematic elements of the concept maps. Statistically significant correlations are then identified by treating the background and academic data as independent variables and the concept map structural and thematic metrics as dependent variables. The findings are discussed in the context of their fundamental contributions to knowledge about student learning as well as their implications to support engineering design education improvement.

## 2 Background

To provide a framework for the analysis, this section presents an examination of the existing literature on design education, the general impact of unique student backgrounds on learning, and concept mapping as a tool for assessing student knowledge.

### 2.1 Engineering Design Education.

Engineering design education has undergone substantial changes over the last 50 years, as studies began to indicate a skills gap in trained engineers [1214]. One early documented effort to address this skills gap was a longitudinal study comparing active and cooperative classroom styles against traditional lecture teaching [15]. Students in an experimental group were introduced to broader concepts beyond their specific field of study that helped them understand course material in better context and create a more robust, applicable expertise of course material. Another example of this was the introduction and growth of capstone design projects and PrBL methods [5]. These types of courses have shown increasing effectiveness on students’ academic achievement in recent years, and by practicing these skills in various contexts in school, students are more prepared to enter the workforce. One study followed several students into their careers following their completion of engineering programs, and surveyed them on their work activities, revealing that team meetings, project management, and refining designs based on customer needs are all key aspects in industry covered by project-based and holistic design curricula [16].

The findings of these and other studies have prompted many institutions to develop more holistic experiences in their engineering programs, allowing new engineers to develop and practice their knowledge and skills, and thereby be more prepared to enter the workforce. This includes design-focused curricula, extracurricular grand design challenges, general education requirements, interdisciplinary training, and co-op and internship programs, among others. However, no studies to date have specifically examined the extent to which student backgrounds and experiences influence their knowledge and perceptions about the holistic nature of design.

### 2.2 Influences of Student Backgrounds in Education.

In various contexts, previous studies have analyzed how different backgrounds and environmental factors impact student performance. One study in Indonesia looked at parental education backgrounds of young students learning the English language, showing that higher parental education levels were significantly correlated with better performance on English assessments [8]. This study indicated that parental support is an influencing factor in education output, and it also suggested that there are other factors that may contribute to student success, including teachers, friends, and environmental factors. On a broader level, students with a role model in the same field of study have been shown to have a better sense of belonging and interest in the subject matter [1719]. For engineering education in particular, studies have found that parents with engineering backgrounds can play a major role in shaping attitudes and motivating their children to study engineering, and, in many cases, can help support their children in learning engineering concepts and skills [20]. As such, whether or not a student had or has a role model in engineering may be a factor that contributes to their understanding of product design.

Prior studies have also explored whether the academic backgrounds of students had an impact on how they approached STEM courses. In relation to academic performance, one study tested interactive teaching methods with two groups of students: one with strong science backgrounds and one with little to no scientific backgrounds [9]. The results of the study found that while both groups of students responded positively to the more active and interactive teaching methods, it was the less-experienced students that saw the greatest improvement in performance. Research has also looked into how a student’s academic major or field of study would impact the product designing process [21]. This study, conducted between three Portuguese institutions similar but independent product design and development master’s courses, found that (1) engineering school students iterated technical and conceptual developments to refine their product’s function, (2) design school students focused on improving the esthetic their original design, while (3) business school students focused on developing the product for marketability. These studies indicate that the program of study would likewise be a key factor in shaping how a student approaches product design.

Research has also investigated the impact of meaningful work experience on student thinking and career preparation. One study measured the career maturity of Australian high school students, some who had paid work experience and some who did not have such experiences [22]. Career maturity measures a student’s readiness to make appropriate career decisions and manage critical tasks associated with career success [23,24]. The study found that students with paid work experience have consistently higher career development attitude scores than those without. The results also suggest that paid work experience can be associated with increased thoughtfulness in career maturity [22]. A more recent study of American university students investigated the impact of internship experiences on their attitudes toward socially responsible engineering (SRE) [25]. Using interviews spanning a four-year period, they found that internships had mixed influences on students’ understandings of SRE, with most of the participants exhibiting minimal change. Another study compared the abilities of students to break-down complex problems against the abilities of experienced professionals, finding that more practical experience leads to a more comprehensive break-down of concepts [26], results that can be graphically represented using concept mapping as a tool.

The study reported here captures similar background factors as the reviewed studies (academic major and plans, meaningful work experience, and presence of engineering role models), in order to identify whether and how these factors influence the ways that students think about product design.

### 2.3 Concept Mapping.

Among the various tools for knowledge representation and assessment, concept mapping is an open-ended approach that allows people to explore their thoughts surrounding a central topic in an unstructured manner, and it has been used by many to study multidisciplinary topics such as product design [2729]. Concept maps are organizational tools similar to flowcharts, but more limited in that they only contain one class of nodal elements (concepts). They are constructed by creating nodes consisting of nouns or noun phrases, and then connecting those nodes together with linking verb phrases [30]. Typically, concept maps originate with a focus question or topic and branch outward. For example, Fig. 1 provides an example concept map that starts with the central concept of product design. Through concept mapping, individuals or groups can express and organize complex connections between different ideas in their minds, ultimately developing a more holistic, robust understanding.

Fig. 1
Fig. 1
Close modal

As a concept map grows, nodes and linking phrases begin to link with concepts from other fields of knowledge. The value of concept maps stems from their ability to display interdisciplinary relationships among concepts. Practicing concept mapping regularly has also been shown to increase effectiveness in learning for young students [31], and in many contexts, concept mapping can also act as an alternative to exams and other more traditional evaluation methods [32]. In the scope of teaching engineering design, the generation of concept maps has been used to map out student understanding and exploration of complexity, making it a valuable tool for evaluating student knowledge acquisition [33,34].

Concept maps are evaluated and assessed differently than more traditional learning evaluation methods. For students, the process of creating concept maps is a powerful method to synthesize knowledge, as it graphically displays and organizes knowledge of a student’s thoughts surrounding a particular concept or field [35]. In the literature and in practice, concept maps are analyzed in many different ways, depending on the purpose of the exercise. Generally, numerically assessing node counts and looking at the network density is a common approach to understanding and evaluating concept maps from a structural perspective [36]. In the context of studying the progression of students over time, a greater number of relationships between nodes has been found to be an indicator of more comprehensive understanding [37]. In many experiments, concept maps are evaluated by comparing to a master map, which includes concepts and links that align with the viewpoints of subject matter experts. Student-generated maps are then compared against these master maps to evaluate thoroughness of understanding [38,39]. While these methods have been proven useful in other studies, the study reported here differs in that the analyses do not include a desired outcome or expert map. This is because design is inherently ubiquitous and context-driven, with no absolutely correct approach [40].

## 3 Methods

This study analyzes data from surveys and concept maps generated on the first day of a third-year engineering design course. The survey asked questions regarding students’ backgrounds coming into the course, and the concept maps mapped out the students’ conceptualizations of product design. The survey data were then compared with the concept map contents to explore correlations between backgrounds and conceptualizations.

### 3.1 Course Context.

The study was conducted at Stevens Institute of Technology, a private STEM university located in the northeastern United States. All undergraduate engineering students at Stevens follow the University’s Design Spine course progression: This is an eight-course series through which students learn and apply different aspects of design in conjunction with other engineering topics. The first five courses are project-based and focus on general engineering topics such as mechanics, dynamics, and materials. The sixth course, Engineering Design VI, is discipline-specific and is the final course of the Design Spine before students begin the year-long capstone design project. This course brings together topics from previous course in a PrBL experience that mirrors the process students will go through in their capstone project, with more emphasis on instruction and guidance.

The participants of this study were all entering their third-year Engineering Design VI course. Survey and concept map data were collected from students in three different disciplines: engineering management (EM), industrial and systems engineering (ISE), and mechanical engineering (ME). The EM and ISE students took this course together in one combined section, and their curricula in the first two years of the program contain substantial overlaps; therefore, their concept maps were grouped together.

### 3.2 Data Collection.

Survey and concept map data were collected from 154 students (125 ME and 29 EM/ISE) during the first week of the third-year Engineering Design VI course. The survey asked about the students’ backgrounds and experiences, and the concept maps were generated around the students’ internal conceptions of “product design.” The data instruments were approved by the Stevens Institutional Review Board under protocol 2017-016(21-R1).

#### 3.2.1 Surveys.

A brief survey was designed to gather data about the students’ backgrounds and experiences. The survey collected data about previous work experience (e.g., internships, co-ops, and research assistantships), intentions regarding whether to pursue a master’s degree, courses that have been completed previously, education level of parents/guardians, and whether they grew up with a parent, guardian, or close adult role model who had an engineering background. Regarding the previous work experience, information was requested about the timing and specific job roles in those work experiences. The complete text of the survey questions and response options are provided in the  Appendix.

#### 3.2.2 Concept Maps.

To measure how students conceptualize product design, they individually generated concept maps around the central theme of “product design.” Prior to constructing these maps, the students were given a brief tutorial on how to construct a concept map, and they constructed a group example concept map on the topic of “personal health.” Following this exercise, they were asked to construct their own using the following prompt:

Draw a concept map that embodies the concept of “product design.” There is no right or wrong answer, as we just want to explore how you think about product design and the factors that are important to consider in product design. Please use the entire 15 minutes to add/revise elements and refine the structure and connections. Remember, concept maps include concepts (in boxes) and relationships (along arrows).

As this course took place during the Spring of 2021 in the midst of the COVID pandemic, the course was held entirely over Zoom. Therefore, the students constructed their concept maps digitally using the lucidchart online diagraming software [41]. The resulting concept maps were submitted, anonymized, and subsequently analyzed. One example from this data set, generated by the participant coded as “E4,” is provided as Fig. 2.

Fig. 2
Fig. 2
Close modal

### 3.3 Data Analysis.

The concept maps were analyzed in two ways: structurally and thematically. The structural analysis viewed each concept map as a quantitative network, looking at the number of nodes, the number of links, and the network density. The thematic analysis involved categorizing the contents of the nodes and evaluating the relative presence of different themes. This resulted in dependent variables for subsequent statistical analyses, and four binary independent variables from the surveys were used to evaluate their predictive capabilities: academic major, work experience, plans to enter a master’s program, and presence of an engineering parent or role model.

#### 3.3.1 Structural Analysis.

The structural dependent variables included in the analysis were node count and network density. The node count is simply the number of concepts the student included in their map. Network density is a ratio of the number of arrows to the maximum potential arrows in a concept map, given the number of nodes. Density (ρ) is calculated using Eq. (1), where e is the number of edges/arrows and n is the number of nodes. The denominator represents the potential links, or the number of edges that the concept map could have if every node were connected to every other node.
$ρ=en(n−1)$
(1)
For example, E4’s concept map in Fig. 2 contains 11 terms/nodes and 16 arrows/edges. If every node were connected both to and from every other node in this map, there would be 110 edges (the denominator in Eq. (1)); therefore, the density ρ of this example is 16/110 = 0.145.

Shapiro–Wilk tests were performed to determine whether the response variables are normally distributed, and they concluded that the data are non-normal. Therefore, non-parametric Mann–Whitney U tests were performed for pairwise comparisons of each dependent variable with respect to the four categorical independent variables (academic major, work experience, plans to enter a master’s program, and presence of an engineering parent or role model) [42]. These tests identified whether each independent variable had a significant influence on the dependent variable. Furthermore, interaction effects were examined using Kruskal–Wallis H tests to see whether there were significant effects from combinations of independent variables that might have otherwise been missed in the U tests. The resulting analysis identifies which factors significantly influenced the structure of the concept maps and to what extent.

#### 3.3.2 Thematic Analysis.

In addition to analyzing the structure of the concept maps, it was critical to also look into the themes present. One of the most common methods of evaluating concept maps is to identify the presence of certain root themes and terms within the maps [43]. When analyzing concept map content in engineering design contexts, there are a variety of different methods. Some research indicates that words should be broadly categorized into three buckets: technology, business, and people [44]. Other researchers have taken a more specific approach, categorizing words in more specific themes including things like design knowledge, theory, and finance [45]. In the study reported here, these two methods were combined, allowing researchers to search for the presence of broad themes and also specific categorical terms.

In a previous study as part of this project [46], the terms that appeared in the product design concept maps were categorized using an inductive coding process. Each term that appeared as a node in a concept map was examined independently by two researchers, who manually categorized the terms into an evolving list of themes and sub-themes that captured the breadth of content. These classifications were checked and revised multiple times by the research team until a consensus categorization scheme was reached, following a process similar to Rye and Rubba [47]. This resulted in three thematic areas, each with four associated sub-themes, summarized in Table 1.

Table 1

Three major themes and their four respective sub-themes

Technical skillsFinanceGovernance
Conceptual developmentMarketSustainability
Prototyping and testingOperationsEthics
Manufacturing and productionProject managementStandards and codes
Technical skillsFinanceGovernance
Conceptual developmentMarketSustainability
Prototyping and testingOperationsEthics
Manufacturing and productionProject managementStandards and codes

Once the dictionary of terms with their themes and sub-themes was complete, the percentage of terms in a given concept map in each theme and sub-theme is calculated. For example, if a concept map has ten total terms, and three of them were categorized as engineering, the resulting engineering term ratio is 0.30. For the concept map example in Fig. 2, four of the 11 terms were classified as engineering (brainstorming, prototypes, solution, and testing), five were classified as business (competition, marketing, implementation or sales, resources, and user feedback), and the other two (product design and problem) were deemed too broad to categorize. This concept map has an engineering thematic term ratio of 0.364, a business term ratio of 0.455, and a society term ratio of 0. The sub-thematic ratios in this map are conceptual development 0.182 (brainstorming and solution), prototyping and testing 0.182 (prototypes and testing), finance 0.091 (implementation or sales), market 0.273 (competition, marketing, and user feedback), and project management 0.091 (resources).

These thematic and sub-thematic ratios were used as dependent variables in the Mann–Whitney U tests. The goal was to identify which, if any, of the background factors led to significant differences in the ratios of specific themes and sub-themes.

### 3.4 Limitations.

There are notable limitations to this study. First, the sample size is limited to the 154 students who participated in the study. With four independent variables plus their six interaction effects, this was a limited sample that was constrained by the participant pool. Additionally, since there is no consensus on what specific topics, links, and themes should be present in a “correct” concept map of product design, this analysis does not evaluate the quality of student understanding of product design. Rather, the study provides insight into what types of themes students of different backgrounds include in their maps, and what gaps these students may have in their initial understandings.

## 4 Results

The results show how the structural and thematic contents of student concept maps correlate with a variety of background factors. Using a custom python program leveraging the scipy library, version 1.7.1 [48], individual influences of independent variables were examined, along with interaction effects between every pair of independent variables. Table 2 summarizes the results of 68 Mann–Whitney U tests, showing which of the four independent variables (columns) exhibited significant (p < 0.05) correlations with each of the 17 dependent variables (rows), with the corresponding p-values and Cliff’s delta (δ) measures of effect size when applicable. Cliff’s δ values are commonly interpreted using the following thresholds [49]:

• |δ| < 0.147: insignificant

• |δ| ≥ 0.147: small

• |δ| ≥ 0.33: medium

• |δ| ≥ 0.474: large

Table 2

Experimental parameters with significant effects shown, based on 68 Mann–Whitney U tests

Dependent var./independent var.Academic majorWork experienceMaster’s programRole model
Node count
Network densityp = 0.004, δ = −0.36
Engineering ratiop = 0.025, δ = 0.28
Technical skills ratiop = 0.007, δ = 0.24
Conceptual dev. ratio
Prototyping and testing ratiop = 0.001, δ = 0.39
Mfg. and production ratiop = 0.004, δ = 0.21p = 0.046, δ = −0.15
Business ratiop = <0.001, δ = −0.49
Finance ratiop = 0.036, δ = −0.16
Market ratiop = 0.002, δ = −0.38p = 0.011, δ = 0.26
Operations ratio
Project management ratio
Society ratiop = 0.035, δ = −0.19p = 0.022, δ = 0.21
Governance ratio
Sustainability ratiop = 0.005, δ = −0.23
Ethics ratiop = 0.034, δ = 0.14p = 0.011, δ = −0.14
Standards and codes ratiop = 0.034, δ = −0.10
Total factors influenced7252
Dependent var./independent var.Academic majorWork experienceMaster’s programRole model
Node count
Network densityp = 0.004, δ = −0.36
Engineering ratiop = 0.025, δ = 0.28
Technical skills ratiop = 0.007, δ = 0.24
Conceptual dev. ratio
Prototyping and testing ratiop = 0.001, δ = 0.39
Mfg. and production ratiop = 0.004, δ = 0.21p = 0.046, δ = −0.15
Business ratiop = <0.001, δ = −0.49
Finance ratiop = 0.036, δ = −0.16
Market ratiop = 0.002, δ = −0.38p = 0.011, δ = 0.26
Operations ratio
Project management ratio
Society ratiop = 0.035, δ = −0.19p = 0.022, δ = 0.21
Governance ratio
Sustainability ratiop = 0.005, δ = −0.23
Ethics ratiop = 0.034, δ = 0.14p = 0.011, δ = −0.14
Standards and codes ratiop = 0.034, δ = −0.10
Total factors influenced7252

Note: Empty cells indicate no significant correlation, and numbers represent the p-values of significant effects (p < 0.05) with corresponding Cliff’s delta (δ) measures of effect size; effect sizes are considered small when |δ| > 0.147, medium when |δ| > 0.33, and large when |δ| > 0.474

For example, the box toward the upper left of Table 2 showing a p-value of 0.004 is the medium-significance result of a Mann–Whitney U test comparing the concept map densities of students in the ME major versus those of EM/ISE students. Of the independent variables, academic major was a significant factor in the highest number of dependent variables (seven). While interaction effects were explored using Kruskal–Wallis H tests, the only significant results were in areas where there was already a known significant variable from the Mann–Whitney U tests. So, these interaction results are not presented or discussed.

### 4.1 Structural Analysis Findings.

The structural analysis revealed only one significant difference among the eight examined, specifically pointing out a medium difference in the concept map network densities of students in the ME versus the EM/ISE programs. Shown in Fig. 3, the EM/ISE students produced significantly denser concept maps than the ME students.

Fig. 3
Fig. 3
Close modal

### 4.2 Thematic Analysis Findings.

Within the three major themes—engineering, business, and society—there were four significant findings from the analyses. The one with the largest effect size (δ = −0.49) is the influence of academic major on the business theme, illustrated in Fig. 4. As might be expected, students enrolled in the EM and ISE majors, which include management and operations topics in the curricula, consider business-related topics in their mental models of product design with higher frequency than students in ME majors.

Fig. 4
Fig. 4
Close modal

On the other hand, ME students considered engineering terms with more frequency than EM/ISE students, seen in Fig. 5. As these three thematic ratios have a zero sum, this is where the ME students have shifted their focus; however, the effect size on this observation is small. Two different background factors had small significant influences on the society ratio in student concept maps: presence of a role model and intentions to pursue a master’s program. Students with engineering role models in their lives tended to include more society terms in their concept maps, see Fig. 6, as did students who were not pursuing a master’s degree, see Fig. 7.

Fig. 5
Fig. 5
Close modal
Fig. 6
Fig. 6
Close modal
Fig. 7
Fig. 7
Close modal

### 4.3 Sub-thematic Analysis.

Because the sub-thematic level includes 12 categories, there are inherently fewer words per category than with the broader themes. There is also a wider array of usage with sub-themes. Some sub-themes were included often, while some were rare. For example, the average student included over 20% conceptual development terms in their maps, whereas the average student only included 0.2% governance terms. This is a likely explanation for why there were no statistically significant results in the latter category.

The remaining 11 significant Mann–Whitney U tests from Table 2 were associated with the sub-thematic response variables. Much like in the thematic analysis, students’ field of study influences several sub-thematic categories. The most significant are their ratios of prototyping and testing terms (see Fig. 8), market terms (see Fig. 9), and technical skills terms (see Fig. 10). Students majoring in ME used significantly more terms associated with prototyping and testing than those in the EM and ISE majors, with a medium effect size. ME students also used more technical skills terms than EM/ISE students, though it should be noted that the majority of students in both samples used no technical skills terms, as the median is zero; still, ME students were much more likely to include these terms. EM/ISE students, however, had substantially more market terms in their conceptual models. This can be viewed as a deeper dive into the trend from Fig. 4, as market is a sub-theme within the business theme.

Fig. 8
Fig. 8
Close modal
Fig. 9
Fig. 9
Close modal
Fig. 10
Fig. 10
Close modal

Master’s program intentions had significant effects on students’ inclusion of sustainability and market terms, seen in Figs. 11 and 12. Those planning to pursue a master’s degree tended to include more market and fewer sustainability terms than those intending to seek an industry job after the bachelor’s degree. Interestingly, when examining the influence of student work experience, there was a significant effect on the inclusion of finance terms, shown in Fig. 13, where those with substantive work experience included fewer finance terms than those without. This effect is small, though, as the majority of students in both categories included no finance terms in their concept maps.

Fig. 11
Fig. 11
Close modal
Fig. 12
Fig. 12
Close modal
Fig. 13
Fig. 13
Close modal

## 5 Discussion

The results provide insights into the primary research question posed at the beginning of this article: How do the backgrounds and academic profiles of third-year engineering students—specifically, their academic major and whether they have meaningful work experience, grew up with an engineering role model, and intend to pursue a master’s degree—influence their conceptions of product design, as measured through the breadth and thematic contents of individually generated concept maps? The statistical analyses revealed some differences among student conceptual models of design based on their academic major, intentions to pursue a master’s degree, meaningful work experience, and presence of an engineering role model in their lives.

### 5.1 Degree Programs and Design Conceptions.

The most significant of these factors, both in terms of frequency and effect size, is found to be the academic major. Students enrolled in the ME degree program included more terms associated with the engineering theme, particularly in the sub-themes of prototyping and testing and technical skills. The other group of students, who were enrolled in the EM and ISE programs, included denser concept maps with more business terms, particularly in the market sub-theme.

These differences across majors may be explained by the varying course curricula (prior to the Engineering Design VI course) between ME and EM/ISE students, as well as the predispositions of students who choose to pursue these major fields of study. By the time students reach their sixth academic term, EM and ISE students have taken courses such as project management, accounting and business analysis, and logistics and supply chain management, whereas ME students have taken courses such as fluid mechanics, design of machine components, and ME thermodynamics. When only looking at student majors, these course differences may explain the observed disparity between thematic and sub-thematic ratios. Additionally, the EM and ISE curricula include courses and content on complex systems and systems modeling, which may explain their tendancy to see more interconnections among their map elements (i.e., higher density). Another explanation for these findings is that students choosing to study ME are more inclined to focus on the technical engineering topics, whereas those choosing EM and ISE tend to think more about the broader system, including non-engineering factors.

These findings reinforce the observations of Silva et al. [21], who found that students in product development courses tend to focus their project work on particular areas associated with their fields of study. In their study, engineering students focused on engineering aspects like prototyping and technical development, and business students focused on marketability. Similarly, in the study reported here, ME students were found to think more about prototyping and technical skills, while EM students—who often consider themselves to be at the intersection of engineering and business—include relatively more market-related concepts.

Four small effects were found when comparing students who intend to pursue a master’s degree against those who do not; three of these effects are shown in Figs. 7, 11, and 12. Those who seek to continue in a master’s program included, on average, more market and manufacturing and production sub-theme terms and fewer society terms, particularly in the sub-theme of sustainability. One possible explanation is that students seeking to go into industry are more concerned about making societal and sustainability-related impacts, whereas those intending to continue their studies are more cognizant of complex business models and market factors, including marketing themselves for higher-level corporate positions.

### 5.2 Backgrounds and Design Conceptions.

Two of the independent variables studied—meaningful work experience and presence of an engineering role model—concerns students’ individual backgrounds and experiences. For each of these factors, only two of the 17 dependent variables were found to be affected, with small effect sizes at best. Regarding the presence of an engineering role model, those students who did have a role model tended to include more society terms and fewer manufacturing and production terms. Role models in an academic field, particularly in engineering, are known to shape a student’s sense of belonging, attitude, and motivation toward that field [1719]. The results of the study reported here show specifically that such role models help students consider the context of society when thinking about product design. This may come at the expense of considering the specific downstream engineering concepts of manufacturing and production, though the effect size is smaller.

The second of these variables is whether a student has meaningful work experience in the form of an internship or co-op. Interestingly, students with such work experience were found to use fewer terms associated with finance and ethics. This lack of focus on non-technical aspects of design may be related to the type of siloed work that undergraduate engineering student interns are assigned. Further research is needed to dissect the students based on the type of work experience they performed in companies. The lack of significant results associated with this variable mirrors the results of Rulifson and Bielefeldt [25], who found mixed influences of internships on understanding of contextual factors.

### 5.3 Recommendations for Future Work.

The findings provide numerical evidence about the ways that students think about product design as they enter a sixth-semester engineering design course. In particular, they shed light on how background factors and academic degree programs may specifically influence the way students think about design, which allows course and curriculum designers to better understand how different student populations have different needs and gaps in their knowledge when beginning a course on holistic engineering design. As the results may only be interpreted within the context of one university and a relatively homogenous sample of students, it also provides a methodological framework for others to follow, both in the context of assessing prior knowledge of students in a classroom and in the context of engineering education research.

This article supports the need to further analyze students’ prior experiences, both within their academic programs and outside the classroom. One recommendation for instructors of future design courses is to collect data at the beginning of the course, and then tailor the course syllabus to the gaps in student conceptual models. This could be done in a comprehensive way through concept map collection and analysis, as was done in this study. However, this is time intensive, and so instructors may more easily survey their students about their backgrounds and infer learning needs from the correlations revealed in this and other similar studies. Such actions may lead to students who are better able to put their technical training into context, and institutions will build a stronger, more well-rounded pipeline of students who can approach design problems in holistic ways.

This research creates a foundation upon which further studies may build. The methodology and findings presented in this paper reveal several opportunities to supplement and expand on this domain. As the study took place in three programs at one private institution with a high proportion of white students, one direction is to expand to a more diverse population. Including students from different institutions and in different fields of study would yield more robust results that may provide additional support to generalize (or differentiate against) the findings in this paper. Furthermore, future research may include more depth in the demographic, background, and academic profile variables. In the study reported here, each independent variable was binary (e.g., yes or no, ME or EM/ISE). However, there are further details about the students which could be expanded into additional or more complex independent variables (e.g., type of work experience, education level of parents/guardians). While the sample size in the present study made this unlikely to produce statistically meaningful results, as the subsets of students would be quite small, a larger sample may make such a follow-up study more suitable. This would also open the door to include additional types of data that extend beyond the binary variables included in this article.

Lastly, an opportunity is presented to further refine the methods by which the concept maps are analyzed. Through more advanced network analysis strategies and/or concept map analysis tools, further research may uncover additional findings beyond the dependent variables utilized in this study. Two specific ideas are to investigate specific node pairings and to research trends in the ways certain themes connect with other themes within the concept maps. Furthermore, research to rigorously develop an industry-based expert concept map around the topic of product design could enable a dependent variable that measures concept map quality in a meaningful way, which can enable conclusions regarding which groups of students are better prepared for careers in industry.

## 6 Conclusion

This study took an exploratory approach to identify in what ways student backgrounds and academic degree programs may influence their conceptions of product design. Four academic and background factors were considered: academic major, intentions to pursue a master’s degree, internship or co-op experience, and the presence of an engineering role model in their lives. The most influential factor was academic major. Students in the ME program tended to consider more engineering factors such as prototyping, testing, and technical skills, whereas those in the EM and ISE programs considered more business-related aspects of design such as market factors. The EM and ISE students also had significantly denser concept maps, indicating a tendency to recognize more interconnections among the concepts they included in their concept maps. The other three factors—students’ intentions to pursue a master’s degree, whether they had an engineering role model, and whether they had meaningful work experience—were also indicators of small differences in the thematic contents of their concept maps. These findings provide insights on the gaps in students’ knowledge about holistic product design, the ways that outside factors and experiences may or may not be able to fill those gaps, and a baseline upon which educators can use to design improved engineering curricula for today’s students.

## Acknowledgment

This material is based upon work supported by the U.S. National Science Foundation under Grant Numbers 1927037 and 1927114. Any opinions, findings, and conclusions or recommendations expressed in this material are those of the authors and do not necessarily reflect the views of the National Science Foundation. The authors gratefully acknowledge the support as well as the contributions of the study participants.

## Conflict of Interest

There are no conflicts of interest.

## Data Availability Statement

The datasets generated and supporting the findings of this article are obtainable from the corresponding author upon reasonable request.

## Appendix: Survey Text and Response Options

1. Which of the following work experiences have you had in a technical role? (Check all that apply)

• Summer internship, this past summer (2020)

• Summer internship, previous summer (2019 or earlier)

• Co-op

• Co-op (select this if you’ve done more than 1 co-op)

• Research with a faculty member at Stevens

• Research at another institution

• None of the above

2. Please list the companies that you have worked for in internship or co-op positions

3. What other (technical or non-technical) jobs have you held that do not fit the above categories? Please list the role and company

4. What was your primary role in your internship and co-op positions? (Check all that apply)

• Project management or scheduling

• Technical design

• Non-technical design

• Manufacturing

• Logistics and supply chain management

• Data analytics

• Finance

• I have not had internship or co-op experiences

5. Are you planning to complete a master’s degree at Stevens? (Mark only one)

• Yes, I am in or considering the Accelerated Master’s Program (AMP) or 4+1 program

• Yes, but not through the AMP or 4+1 program

• Possibly

• No

6. If you are completing or considering a master’s at Stevens, in what discipline will it be?

7. What courses are you currently taking (Spring 2021 term)? (Check all that apply)

• (Included list of typical major-specific courses)

8. Which of the following courses have you already taken (BEFORE Spring 2021 term)?

• (Included list of typical major-specific courses)

9. What is the highest level of education of your parents/guardians? (Choose the highest level among your parents/guardians)

• No formal education

• High school diploma or GED

• College degree

• Vocational training

• Bachelor’s degree

• Master’s degree

• Professional degree

• Doctorate degree

• Unsure/prefer not to stay

10. Did you grow up with a parent, guardian, or close adult role model who has/had an engineering backgrounds? (Check all that apply)

• Yes, at least one with an engineering degree

• Yes, at least one with experience working as an engineer

• Yes, at least one with engineering research experience

• Yes, more than one with some engineering degree or work experience

• Not sure

• No

• Female

• Male

• Non-binary

• Prefer not to say

• Other

## References

1.
Sheppard
,
K.
, and
Gallois
,
B.
,
1999
, “
The Design Spine: Revision of the Engineering Curriculum to Include a Design Experience Each Semester
,”
ASEE Annual Conference Proceedings
,
Charlotte, NC
,
June 20–23
, pp.
4887
4893
.
2.
Crawley
,
E. F.
,
Malmqvist
,
J.
,
Östlund
,
S.
,
Brodeur
,
D. R.
, and
Kristina
,
E.
,
2014
,
Rethinking Engineering Education: The CDIO Approach
, 2nd ed.,
Springer
,
Cham
.
3.
Sanders
,
M. E.
,
2012
, “Integrative STEM Education as ‘Best Practice’,”
Explorations of Best Practice in Technology, Design, & Engineering Education
, Vol.
2
,
Middleton
,
H.
, ed.,
Griffith Institute for Educational Research
,
Mt. Gravatt, Australia
, pp.
103
117
.
4.
Howe
,
S.
, and
Wilbarger
,
J.
,
2006
, “
2005 National Survey of Engineering Capstone Design Courses
,”
2006 Annual ASEE Conference & Exposition
,
Chicago, IL
,
June 18–21
.
5.
Chen
,
C.-H.
, and
Yang
,
Y.-C.
,
2019
, “
Revisiting the Effects of Project-Based Learning on Students’ Academic Achievement: A Meta-analysis Investigating Moderators
,”
Educ. Res. Rev.
,
26
, pp.
71
81
.
6.
Mejtoft
,
T.
,
2016
, “
Integrating Business Skills in Engineering Education: Enhancing Learning Using a CDIO Approach
,”
The 12th International CDIO Conference: Proceedings—Full Papers
,
Turku, Finland
,
June 12–16
.
7.
Lee
,
O.
,
2003
, “
Equity for Culturally and Linguistically Diverse Students in Science Education: Recommendations for a Research Agenda
,”
Teach. Coll. Rec.
,
105
(
3
), pp.
465
489
.
8.
Marzulina
,
L.
,
Pitaloka
,
N. L.
,
Herizal
,
H.
,
Holandyah
,
M.
,
Erlina
,
D.
, and
Lestari
,
I. T.
,
2018
, “
Looking at the Link Between Parents’ Educational Backgrounds and Students’ English Achievement
,”
Indonesian Res. J. Educ.
,
2
(
1
), pp.
51
60
.
9.
Ernst
,
H.
, and
Colthorpe
,
K.
,
2007
, “
The Efficacy of Interactive Lecturing for Students With Diverse Science Backgrounds
,”
,
31
(
1
), pp.
41
44
.
10.
Strauss
,
L. C.
, and
Terenzini
,
P. T.
,
2007
, “
The Effects of Students’ In- and Out-of-Class Experiences on Their Analytical and Group Skills: A Study of Engineering Education
,”
Res. High. Educ.
,
48
(
8
), pp.
967
992
.
11.
Ozek
,
H. Z.
,
2018
, “
Impact of Internship Programme in Engineering Education
,”
Eurasia Proc. Educ. Soc. Sci.
,
9
, pp.
276
283
.
12.
Brunhaver
,
S. R.
,
Korte
,
R. F.
,
Barley
,
S. R.
, and
Sheppard
,
S. D.
,
2018
, “Bridging the Gaps Between Engineering Education and Practice,”
U.S. Engineering in a Global Economy
,
Freeman
,
R. B.
, and
Salzman
,
H.
, eds.,
University of Chicago Press
,
Chicago, IL
, pp.
129
163
.
13.
May
,
E.
, and
Strong
,
D. S.
,
2006
, “
Is Engineering Education Delivering What Industry Requires
,”
Proceedings of the Canadian Design Engineering Network (CDEN) Conference
,
,
July 24–26
.
14.
Male
,
S. A.
,
Bush
,
M. B.
, and
Chapman
,
E. S.
,
2010
, “
Perceptions of Competency Deficiencies in Engineering Graduates
,”
Tech. Rep. 1
,
The University of Western Australia
,
Perth
.
15.
Felder
,
R.
,
Felder
,
G.
, and
Dietz
,
E.
,
1998
, “
A Longitudinal Study of Engineering Student Performance and Retention. V. Comparisons With Traditionally-Taught Students
,”
J. Eng. Educ.
,
87
(
4
), p.
10
.
16.
Gewirtz
,
C.
,
Kotys-Schwartz
,
D. A.
,
Knight
,
D.
,
Paretti
,
M. C.
,
Arunkumar
,
S.
,
Ford
,
J. D.
,
Howe
,
S.
,
Rosenbauer
,
L. M.
,
Alvarez
,
N. E.
,
Deters
,
J.
, and
Hernandez
,
C.
,
2018
, “
New Engineers’ First Three Months: A Study of the Transition From Capstone Design Courses to Workplaces
,”
2018 ASEE Annual Conference & Exposition, ASEE Conferences
,
Salt Lake City, UT
,
June 24–27
.
17.
Shin
,
J. E. L.
,
Levy
,
S. R.
, and
London
,
B.
,
2016
, “
Effects of Role Model Exposure on Stem and Non-stem Student Engagement
,”
J. Appl. Soc. Psychol.
,
46
(
7
), pp.
410
427
.
18.
Neumark
,
D.
, and
Gardecki
,
R.
,
1998
, “
Women Helping Women? Role-Model and Mentoring Effects on Female Ph. D. Student in Economics
,”
J. Hum. Resour.
,
33
(
1
), pp.
220
246
.
19.
Ashworth
,
J.
, and
Evans
,
J. L.
,
2001
, “
Modeling Student Subject Choice at Secondary and Tertiary Level: A Cross-Section Study
,”
J. Econ. Educ.
,
32
(
4
), pp.
311
320
.
20.
Dorie
,
B. L.
,
Jones
,
T. R.
,
Pollock
,
M. C.
, and
Cardella
,
M. E.
,
2014
, “
Parents as Critical Influence: Insights From Five Different Studies
,”
ASEE Annual Conference & Exposition
,
Indianapolis, IN
,
June 15–18
, pp.
924
968
.
21.
Silva
,
A.
,
Leite
,
M.
,
Vilas-Boas
,
J.
, and
Simões
,
R.
,
2019
, “
How Education Background Affects Design Outcome: Teaching Product Development to Mechanical Engineers, Industrial Designers and Managers
,”
Eur. J. Eng. Educ.
,
44
(
4
), pp.
545
569
.
22.
Creed
,
P. A.
, and
Patton
,
W.
,
2003
, “
Differences in Career Attitude and Career Knowledge for High School Students With and Without Paid Work Experience
,”
Int. J. Educ. Vocat. Guid.
,
3
(
1
), pp.
21
33
.
23.
Savickas
,
M.
,
1984
, “
Career Maturity: The Construct and Its Measurement
,”
Vocat. Guid. Quart.
,
32
(
6
), pp.
222
231
.
24.
Creed
,
P.
, and
Patton
,
W.
,
2004
, “
The Development and Validation of a Short Form of the Australian Version of the Career Development Inventory
,”
J. Psychol. Counsellors Schools
,
14
(
12
), pp.
125
138
.
25.
Rulifson
,
G.
, and
Bielefeldt
,
A.
,
2018
, “
Influence of Internships on Engineering Students’ Attitudes About Socially Responsible Engineering
,”
IEEE Frontiers in Education Conference
,
San Jose, CA
,
Oct. 3–6
,
IEEE
, pp.
1
6
.
26.
Ball
,
L. J.
,
Ormerod
,
T. C.
, and
Morley
,
N. J.
,
2004
, “
Spontaneous Analogising in Engineering Design: A Comparative Analysis of Experts and Novices
,”
Des. Stud.
,
25
(
5
), pp.
495
508
.
27.
Walker
,
J. M. T.
,
Cordray
,
D. S.
,
King
,
P. H.
, and
Fries
,
R. C.
,
2005
, “
Expert and Student Conceptions of the Design Process: Developmental Differences With Implications for Educators
,”
Int. J. Eng. Educ.
,
21
(
3
), pp.
467
479
.
28.
Besterfield-Sacre
,
M.
,
Gerchak
,
J.
,
Lyons
,
M. R.
, and
Shuman
,
L. J.
,
2004
, “
Scoring Concept Maps: An Integrated Rubric for Assessing Engineering Education
,”
J. Eng. Educ.
,
93
(
2
), pp.
105
115
.
29.
Segalas
,
J.
,
Ferrer-Balas
,
D.
, and
Mulder
,
K.
,
2008
, “
Conceptual Maps: Measuring Learning Processes of Engineering Students Concerning Sustainable Development
,”
Eur. J. Eng. Educ.
,
33
(
3
), pp.
297
306
.
30.
Novak
,
J.
, and
Cañas
,
A.
,
2007
, “
Theoretical Origins of Concept Maps, How to Construct Them, and Uses in Education
,”
Reflect. Educ.
,
3
(
1
), pp.
29
42
.
31.
Hwang
,
G.-J.
,
Kuo
,
F.-R.
,
Chen
,
N.-S.
, and
Ho
,
H.-J.
,
2014
, “
Effects of an Integrated Concept Mapping and Web-Based Problem-Solving Approach on Students’ Learning Achievements, Perceptions and Cognitive Loads
,”
Comput. Educ.
,
71
, pp.
77
86
.
32.
Walker
,
J.
, and
King
,
P.
,
2002
, “
Concept Mapping as a Form of Student Assessment and Instruction in the Domain of Bioengineering
,”
J. Eng. Educ.
33.
Ferguson
,
S. M.
,
Foley
,
R. W.
,
Eshirow
,
J. K.
, and
Pollack
,
C. C.
,
2018
, “
Refining Concept Maps as Method to Assess Learning Outcomes Among Engineering Students
,”
2018 ASEE Annual Conference & Exposition
,
Salt Lake City, UT
,
June 24–27
.
34.
Sims-Knight
,
J.
,
Upchurch
,
R.
,
Pendergrass
,
N.
,
Meressi
,
T.
,
Fortier
,
P.
,
Tchimev
,
P.
,
VonderHeide
,
R.
, and
Page
,
M.
,
2004
, “
Using Concept Maps to Assess Design Process Knowledge
,”
34th Annual Frontiers in Education, 2004, FIE 2004
,
Savannah, GA
,
Oct. 20–23
, p.
F1G
6
.
35.
Ruiz-Primo
,
M.
, and
Shavelson
,
R.
,
1996
, “
Problems and Issues in the Use of Concept Maps in Science Assessment
,”
J. Res. Sci. Teach.
,
33
(
8
), pp.
569
600
.
36.
Jacobs-Lawson
,
J. M.
, and
Hershey
,
D. A.
,
2002
, “
Concept Maps as an Assessment Tool in Psychology Courses
,”
Teach. Psychol.
,
29
(
1
), pp.
25
29
.
37.
Novak
,
J. D.
,
1990
, “
Concept Maps and VEE Diagrams: Two Metacognitive Tools to Facilitate Meaningful Learning
,”
Instruct. Sci.
,
19
(
1
), pp.
29
52
.
38.
Williams
,
C. G.
,
1998
, “
Using Concept Maps to Assess Conceptual Knowledge of Function
,”
J. Res. Math. Educ.
,
29
(
4
), pp.
414
421
.
39.
Takeya
,
M.
,
Sasaki
,
H.
,
Nagaoka
,
K.
, and
Yonezawa
,
N.
,
2004
, “
A Performance Scoring Method Based on Quantitative Comparison of Concept Maps by a Teacher and Students
,”
Proceedings of the First International Conference on Concept Mapping
,
Pamplona, Spain
,
Sept. 14–17
.
40.
Simpson
,
T.
,
Barton
,
R.
, and
Celento
,
D.
,
2008
, “
Interdisciplinary by Design
,”
Mech. Eng.
,
130
(
9
), pp.
30
33
.
41.
Lucid Software, Inc.
,
2022
, “
Intelligent Diagramming: Lucidchart
,” Online, https://www.lucidchart.comhttps://www.lucidchart.com. Accessed February 2, 2022.
42.
Ho
,
R.
,
2006
,
Handbook of Univariate and Multivariate Data Analysis and Interpretation With SPSS
,
Chapman and Hall/CRC
,
Boca Raton, FL
.
43.
Kinchin
,
I. M.
,
Möllits
,
A.
, and
Reiska
,
P.
,
2019
, “
Uncovering Types of Knowledge in Concept Maps
,”
Educ. Sci.
,
9
(
2
), p.
131
.
44.
Stappers
,
P. J.
,
Hekkert
,
P.
, and
Keyson
,
D.
,
2007
, “
Design for Interaction: Consolidating the User-Centred Focus in Industrial Design Engineering
,”
DS 43: Proceedings of E&PDE 2007, The 9th International Conference on Engineering and Product Design Education
,
Sept. 13–14
,
University of Northumbria
,
Newcastle, UK
, pp.
69
74
.
45.
Horvath
,
I.
,
2004
, “
A Treatise on Order in Engineering Design Research
,”
Res. Eng. Des.
,
15
(
3
), pp.
155
181
.
46.
Miska
,
J. W.
,
Mathews
,
L.
,
Driscoll
,
J.
,
Hoffenson
,
S.
,
Crimmins
,
S.
,
Espera Jr.
,
A.
, and
Pitterson
,
N.
,
2022
, “
How Do Undergraduate Engineering Students Conceptualize Product Design? An Analysis of Two Third-Year Design Courses
,”
J. Eng. Educ.
,
111
(
3
), pp.
616
641
.
47.
Rye
,
J. A.
, and
Rubba
,
P. A.
,
1998
, “
An Exploration of the Concept Map as an Interview Tool to Facilitate the Externalization of Students’ Understandings About Global Atmospheric Change
,”
J. Res. Sci. Teach.
,
35
(
5
), pp.
521
546
.
48.
Virtanen
,
P.
,
Gommers
,
R.
,
Oliphant
,
T. E.
,
Haberland
,
M.
,
Reddy
,
T.
,
Cournapeau
,
D.
,
Burovski
,
E.
,
Peterson
,
P.
,
Weckesser
,
W.
,
Bright
,
J.
, et al.,
2020
, “
SciPy 1.0: Fundamental Algorithms for Scientific Computing in Python
,”
Nat. Methods
,
17
, pp.
261
272
.
49.
Romano
,
J.
,
Kromrey
,
J. D.
,
Coraggio
,
J.
, and
Skowronek
,
J.
,
2006
, “
Appropriate Statistics for Ordinal Level Data: Should We Really Be Using T-Test and Cohen’s d for Evaluating Group Differences on the NSSE and Other Surveys
,”
Annual Meeting of the Florida Association of Institutional Research
,
Cocoa Beach, FL
,
Feb. 1–3
, Vol.
177
, p.
34
.