Abstract
In undergraduate engineering programs, recent emphasis has been placed on a more holistic, interdisciplinary approach to engineering education. Some programs now teach product design within the context of the market, extending the curriculum to topics outside of scientific labs and computational analysis. However, it is unknown to what extent engineering students already understand the systems and contextual factors associated with product design, and also what characteristics or experiences have led students to these ways of thinking. This study analyzes survey and concept map data collected from 154 students in a third-year engineering design course. The aim is to understand how student backgrounds and experiences influence their mental models of product design. Data were gathered from surveys on student backgrounds and experiences, along with concept maps that were generated by the students at the beginning of a product design course. The concept maps were analyzed in a quantitative manner for structural and thematic elements. The findings show that several background attributes influence student conceptions of product design. Academic major appeared to have the largest impact on a variety of variables. Additionally, prior work experience, enrollment in a master’s program, and the presence of an engineering role model at home all showed significant impacts on design conceptions. By analyzing and understanding how the unique backgrounds of students lead to differences in thought, educators can adjust their curricula to more effectively teach design concepts to students of various backgrounds and experiences.
1 Introduction
Traditional undergraduate engineering programs emphasize technical knowledge, with courses in mathematics, physics, mechanics, thermodynamics, and other quantitative topics. While these subjects are undoubtedly critical for aspiring engineers, they can often overpower the importance of design education that also includes non-technical factors such as the markets in which designed artifacts must thrive [1]. This has prompted many institutions to reevaluate their design education curricula, making room for a more holistic approach to engineering design that emphasizes both technical skills and business acumen. Examples of design-related engineering education initiatives include the conceive, design, implement, and operate approach [2]; integrative science, technology, engineering, and mathematics (STEM) education [3]; the proliferation of capstone design courses [4], and the rise in project-based learning (PrBL) [5]. Many of today’s engineering students are now receiving some level of training in interdisciplinary design topics that correspond to the business needs of today’s dynamic professional environments [6]. Given the interdisciplinary nature of product design, it is particularly interesting to consider how diversity within the student population can influence the approach of this topic.
It is widely accepted that students’ individual backgrounds and experiences influence their initial knowledge and conceptions surrounding a topic prior to beginning coursework [7–11]. This study focuses specifically on how engineering student backgrounds influence the breadth and depth within their conceptions of product design prior to beginning a course on the topic. The primary research question is: How do the backgrounds and academic profiles of third-year engineering students—specifically, their academic major and whether they have meaningful work experience, grew up with an engineering role model, and intend to pursue a master’s degree—influence their conceptions of product design, as measured through the breadth and thematic contents of individually generated concept maps?
At the beginning of a third-year undergraduate engineering design course, data were collected from 154 students through a survey and a concept mapping activity. The survey gathered details about the students’ backgrounds and academic profiles, and the concept maps were generated individually around the central concept of “product design.” These maps were explored in a quantitative manner, analyzing both the structural and thematic elements of the concept maps. Statistically significant correlations are then identified by treating the background and academic data as independent variables and the concept map structural and thematic metrics as dependent variables. The findings are discussed in the context of their fundamental contributions to knowledge about student learning as well as their implications to support engineering design education improvement.
2 Background
To provide a framework for the analysis, this section presents an examination of the existing literature on design education, the general impact of unique student backgrounds on learning, and concept mapping as a tool for assessing student knowledge.
2.1 Engineering Design Education.
Engineering design education has undergone substantial changes over the last 50 years, as studies began to indicate a skills gap in trained engineers [12–14]. One early documented effort to address this skills gap was a longitudinal study comparing active and cooperative classroom styles against traditional lecture teaching [15]. Students in an experimental group were introduced to broader concepts beyond their specific field of study that helped them understand course material in better context and create a more robust, applicable expertise of course material. Another example of this was the introduction and growth of capstone design projects and PrBL methods [5]. These types of courses have shown increasing effectiveness on students’ academic achievement in recent years, and by practicing these skills in various contexts in school, students are more prepared to enter the workforce. One study followed several students into their careers following their completion of engineering programs, and surveyed them on their work activities, revealing that team meetings, project management, and refining designs based on customer needs are all key aspects in industry covered by project-based and holistic design curricula [16].
The findings of these and other studies have prompted many institutions to develop more holistic experiences in their engineering programs, allowing new engineers to develop and practice their knowledge and skills, and thereby be more prepared to enter the workforce. This includes design-focused curricula, extracurricular grand design challenges, general education requirements, interdisciplinary training, and co-op and internship programs, among others. However, no studies to date have specifically examined the extent to which student backgrounds and experiences influence their knowledge and perceptions about the holistic nature of design.
2.2 Influences of Student Backgrounds in Education.
In various contexts, previous studies have analyzed how different backgrounds and environmental factors impact student performance. One study in Indonesia looked at parental education backgrounds of young students learning the English language, showing that higher parental education levels were significantly correlated with better performance on English assessments [8]. This study indicated that parental support is an influencing factor in education output, and it also suggested that there are other factors that may contribute to student success, including teachers, friends, and environmental factors. On a broader level, students with a role model in the same field of study have been shown to have a better sense of belonging and interest in the subject matter [17–19]. For engineering education in particular, studies have found that parents with engineering backgrounds can play a major role in shaping attitudes and motivating their children to study engineering, and, in many cases, can help support their children in learning engineering concepts and skills [20]. As such, whether or not a student had or has a role model in engineering may be a factor that contributes to their understanding of product design.
Prior studies have also explored whether the academic backgrounds of students had an impact on how they approached STEM courses. In relation to academic performance, one study tested interactive teaching methods with two groups of students: one with strong science backgrounds and one with little to no scientific backgrounds [9]. The results of the study found that while both groups of students responded positively to the more active and interactive teaching methods, it was the less-experienced students that saw the greatest improvement in performance. Research has also looked into how a student’s academic major or field of study would impact the product designing process [21]. This study, conducted between three Portuguese institutions similar but independent product design and development master’s courses, found that (1) engineering school students iterated technical and conceptual developments to refine their product’s function, (2) design school students focused on improving the esthetic their original design, while (3) business school students focused on developing the product for marketability. These studies indicate that the program of study would likewise be a key factor in shaping how a student approaches product design.
Research has also investigated the impact of meaningful work experience on student thinking and career preparation. One study measured the career maturity of Australian high school students, some who had paid work experience and some who did not have such experiences [22]. Career maturity measures a student’s readiness to make appropriate career decisions and manage critical tasks associated with career success [23,24]. The study found that students with paid work experience have consistently higher career development attitude scores than those without. The results also suggest that paid work experience can be associated with increased thoughtfulness in career maturity [22]. A more recent study of American university students investigated the impact of internship experiences on their attitudes toward socially responsible engineering (SRE) [25]. Using interviews spanning a four-year period, they found that internships had mixed influences on students’ understandings of SRE, with most of the participants exhibiting minimal change. Another study compared the abilities of students to break-down complex problems against the abilities of experienced professionals, finding that more practical experience leads to a more comprehensive break-down of concepts [26], results that can be graphically represented using concept mapping as a tool.
The study reported here captures similar background factors as the reviewed studies (academic major and plans, meaningful work experience, and presence of engineering role models), in order to identify whether and how these factors influence the ways that students think about product design.
2.3 Concept Mapping.
Among the various tools for knowledge representation and assessment, concept mapping is an open-ended approach that allows people to explore their thoughts surrounding a central topic in an unstructured manner, and it has been used by many to study multidisciplinary topics such as product design [27–29]. Concept maps are organizational tools similar to flowcharts, but more limited in that they only contain one class of nodal elements (concepts). They are constructed by creating nodes consisting of nouns or noun phrases, and then connecting those nodes together with linking verb phrases [30]. Typically, concept maps originate with a focus question or topic and branch outward. For example, Fig. 1 provides an example concept map that starts with the central concept of product design. Through concept mapping, individuals or groups can express and organize complex connections between different ideas in their minds, ultimately developing a more holistic, robust understanding.
As a concept map grows, nodes and linking phrases begin to link with concepts from other fields of knowledge. The value of concept maps stems from their ability to display interdisciplinary relationships among concepts. Practicing concept mapping regularly has also been shown to increase effectiveness in learning for young students [31], and in many contexts, concept mapping can also act as an alternative to exams and other more traditional evaluation methods [32]. In the scope of teaching engineering design, the generation of concept maps has been used to map out student understanding and exploration of complexity, making it a valuable tool for evaluating student knowledge acquisition [33,34].
Concept maps are evaluated and assessed differently than more traditional learning evaluation methods. For students, the process of creating concept maps is a powerful method to synthesize knowledge, as it graphically displays and organizes knowledge of a student’s thoughts surrounding a particular concept or field [35]. In the literature and in practice, concept maps are analyzed in many different ways, depending on the purpose of the exercise. Generally, numerically assessing node counts and looking at the network density is a common approach to understanding and evaluating concept maps from a structural perspective [36]. In the context of studying the progression of students over time, a greater number of relationships between nodes has been found to be an indicator of more comprehensive understanding [37]. In many experiments, concept maps are evaluated by comparing to a master map, which includes concepts and links that align with the viewpoints of subject matter experts. Student-generated maps are then compared against these master maps to evaluate thoroughness of understanding [38,39]. While these methods have been proven useful in other studies, the study reported here differs in that the analyses do not include a desired outcome or expert map. This is because design is inherently ubiquitous and context-driven, with no absolutely correct approach [40].
3 Methods
This study analyzes data from surveys and concept maps generated on the first day of a third-year engineering design course. The survey asked questions regarding students’ backgrounds coming into the course, and the concept maps mapped out the students’ conceptualizations of product design. The survey data were then compared with the concept map contents to explore correlations between backgrounds and conceptualizations.
3.1 Course Context.
The study was conducted at Stevens Institute of Technology, a private STEM university located in the northeastern United States. All undergraduate engineering students at Stevens follow the University’s Design Spine course progression: This is an eight-course series through which students learn and apply different aspects of design in conjunction with other engineering topics. The first five courses are project-based and focus on general engineering topics such as mechanics, dynamics, and materials. The sixth course, Engineering Design VI, is discipline-specific and is the final course of the Design Spine before students begin the year-long capstone design project. This course brings together topics from previous course in a PrBL experience that mirrors the process students will go through in their capstone project, with more emphasis on instruction and guidance.
The participants of this study were all entering their third-year Engineering Design VI course. Survey and concept map data were collected from students in three different disciplines: engineering management (EM), industrial and systems engineering (ISE), and mechanical engineering (ME). The EM and ISE students took this course together in one combined section, and their curricula in the first two years of the program contain substantial overlaps; therefore, their concept maps were grouped together.
3.2 Data Collection.
Survey and concept map data were collected from 154 students (125 ME and 29 EM/ISE) during the first week of the third-year Engineering Design VI course. The survey asked about the students’ backgrounds and experiences, and the concept maps were generated around the students’ internal conceptions of “product design.” The data instruments were approved by the Stevens Institutional Review Board under protocol 2017-016(21-R1).
3.2.1 Surveys.
A brief survey was designed to gather data about the students’ backgrounds and experiences. The survey collected data about previous work experience (e.g., internships, co-ops, and research assistantships), intentions regarding whether to pursue a master’s degree, courses that have been completed previously, education level of parents/guardians, and whether they grew up with a parent, guardian, or close adult role model who had an engineering background. Regarding the previous work experience, information was requested about the timing and specific job roles in those work experiences. The complete text of the survey questions and response options are provided in the Appendix.
3.2.2 Concept Maps.
To measure how students conceptualize product design, they individually generated concept maps around the central theme of “product design.” Prior to constructing these maps, the students were given a brief tutorial on how to construct a concept map, and they constructed a group example concept map on the topic of “personal health.” Following this exercise, they were asked to construct their own using the following prompt:
Draw a concept map that embodies the concept of “product design.” There is no right or wrong answer, as we just want to explore how you think about product design and the factors that are important to consider in product design. Please use the entire 15 minutes to add/revise elements and refine the structure and connections. Remember, concept maps include concepts (in boxes) and relationships (along arrows).
As this course took place during the Spring of 2021 in the midst of the COVID pandemic, the course was held entirely over Zoom. Therefore, the students constructed their concept maps digitally using the lucidchart online diagraming software [41]. The resulting concept maps were submitted, anonymized, and subsequently analyzed. One example from this data set, generated by the participant coded as “E4,” is provided as Fig. 2.
3.3 Data Analysis.
The concept maps were analyzed in two ways: structurally and thematically. The structural analysis viewed each concept map as a quantitative network, looking at the number of nodes, the number of links, and the network density. The thematic analysis involved categorizing the contents of the nodes and evaluating the relative presence of different themes. This resulted in dependent variables for subsequent statistical analyses, and four binary independent variables from the surveys were used to evaluate their predictive capabilities: academic major, work experience, plans to enter a master’s program, and presence of an engineering parent or role model.
3.3.1 Structural Analysis.
Shapiro–Wilk tests were performed to determine whether the response variables are normally distributed, and they concluded that the data are non-normal. Therefore, non-parametric Mann–Whitney U tests were performed for pairwise comparisons of each dependent variable with respect to the four categorical independent variables (academic major, work experience, plans to enter a master’s program, and presence of an engineering parent or role model) [42]. These tests identified whether each independent variable had a significant influence on the dependent variable. Furthermore, interaction effects were examined using Kruskal–Wallis H tests to see whether there were significant effects from combinations of independent variables that might have otherwise been missed in the U tests. The resulting analysis identifies which factors significantly influenced the structure of the concept maps and to what extent.
3.3.2 Thematic Analysis.
In addition to analyzing the structure of the concept maps, it was critical to also look into the themes present. One of the most common methods of evaluating concept maps is to identify the presence of certain root themes and terms within the maps [43]. When analyzing concept map content in engineering design contexts, there are a variety of different methods. Some research indicates that words should be broadly categorized into three buckets: technology, business, and people [44]. Other researchers have taken a more specific approach, categorizing words in more specific themes including things like design knowledge, theory, and finance [45]. In the study reported here, these two methods were combined, allowing researchers to search for the presence of broad themes and also specific categorical terms.
In a previous study as part of this project [46], the terms that appeared in the product design concept maps were categorized using an inductive coding process. Each term that appeared as a node in a concept map was examined independently by two researchers, who manually categorized the terms into an evolving list of themes and sub-themes that captured the breadth of content. These classifications were checked and revised multiple times by the research team until a consensus categorization scheme was reached, following a process similar to Rye and Rubba [47]. This resulted in three thematic areas, each with four associated sub-themes, summarized in Table 1.
Three major themes and their four respective sub-themes
Engineering | Business | Society |
---|---|---|
Technical skills | Finance | Governance |
Conceptual development | Market | Sustainability |
Prototyping and testing | Operations | Ethics |
Manufacturing and production | Project management | Standards and codes |
Engineering | Business | Society |
---|---|---|
Technical skills | Finance | Governance |
Conceptual development | Market | Sustainability |
Prototyping and testing | Operations | Ethics |
Manufacturing and production | Project management | Standards and codes |
Once the dictionary of terms with their themes and sub-themes was complete, the percentage of terms in a given concept map in each theme and sub-theme is calculated. For example, if a concept map has ten total terms, and three of them were categorized as engineering, the resulting engineering term ratio is 0.30. For the concept map example in Fig. 2, four of the 11 terms were classified as engineering (brainstorming, prototypes, solution, and testing), five were classified as business (competition, marketing, implementation or sales, resources, and user feedback), and the other two (product design and problem) were deemed too broad to categorize. This concept map has an engineering thematic term ratio of 0.364, a business term ratio of 0.455, and a society term ratio of 0. The sub-thematic ratios in this map are conceptual development 0.182 (brainstorming and solution), prototyping and testing 0.182 (prototypes and testing), finance 0.091 (implementation or sales), market 0.273 (competition, marketing, and user feedback), and project management 0.091 (resources).
These thematic and sub-thematic ratios were used as dependent variables in the Mann–Whitney U tests. The goal was to identify which, if any, of the background factors led to significant differences in the ratios of specific themes and sub-themes.
3.4 Limitations.
There are notable limitations to this study. First, the sample size is limited to the 154 students who participated in the study. With four independent variables plus their six interaction effects, this was a limited sample that was constrained by the participant pool. Additionally, since there is no consensus on what specific topics, links, and themes should be present in a “correct” concept map of product design, this analysis does not evaluate the quality of student understanding of product design. Rather, the study provides insight into what types of themes students of different backgrounds include in their maps, and what gaps these students may have in their initial understandings.
4 Results
The results show how the structural and thematic contents of student concept maps correlate with a variety of background factors. Using a custom python program leveraging the scipy library, version 1.7.1 [48], individual influences of independent variables were examined, along with interaction effects between every pair of independent variables. Table 2 summarizes the results of 68 Mann–Whitney U tests, showing which of the four independent variables (columns) exhibited significant (p < 0.05) correlations with each of the 17 dependent variables (rows), with the corresponding p-values and Cliff’s delta (δ) measures of effect size when applicable. Cliff’s δ values are commonly interpreted using the following thresholds [49]:
|δ| < 0.147: insignificant
|δ| ≥ 0.147: small
|δ| ≥ 0.33: medium
|δ| ≥ 0.474: large
Experimental parameters with significant effects shown, based on 68 Mann–Whitney U tests
Dependent var./independent var. | Academic major | Work experience | Master’s program | Role model |
---|---|---|---|---|
Node count | ||||
Network density | p = 0.004, δ = −0.36 | |||
Engineering ratio | p = 0.025, δ = 0.28 | |||
Technical skills ratio | p = 0.007, δ = 0.24 | |||
Conceptual dev. ratio | ||||
Prototyping and testing ratio | p = 0.001, δ = 0.39 | |||
Mfg. and production ratio | p = 0.004, δ = 0.21 | p = 0.046, δ = −0.15 | ||
Business ratio | p = <0.001, δ = −0.49 | |||
Finance ratio | p = 0.036, δ = −0.16 | |||
Market ratio | p = 0.002, δ = −0.38 | p = 0.011, δ = 0.26 | ||
Operations ratio | ||||
Project management ratio | ||||
Society ratio | p = 0.035, δ = −0.19 | p = 0.022, δ = 0.21 | ||
Governance ratio | ||||
Sustainability ratio | p = 0.005, δ = −0.23 | |||
Ethics ratio | p = 0.034, δ = 0.14 | p = 0.011, δ = −0.14 | ||
Standards and codes ratio | p = 0.034, δ = −0.10 | |||
Total factors influenced | 7 | 2 | 5 | 2 |
Dependent var./independent var. | Academic major | Work experience | Master’s program | Role model |
---|---|---|---|---|
Node count | ||||
Network density | p = 0.004, δ = −0.36 | |||
Engineering ratio | p = 0.025, δ = 0.28 | |||
Technical skills ratio | p = 0.007, δ = 0.24 | |||
Conceptual dev. ratio | ||||
Prototyping and testing ratio | p = 0.001, δ = 0.39 | |||
Mfg. and production ratio | p = 0.004, δ = 0.21 | p = 0.046, δ = −0.15 | ||
Business ratio | p = <0.001, δ = −0.49 | |||
Finance ratio | p = 0.036, δ = −0.16 | |||
Market ratio | p = 0.002, δ = −0.38 | p = 0.011, δ = 0.26 | ||
Operations ratio | ||||
Project management ratio | ||||
Society ratio | p = 0.035, δ = −0.19 | p = 0.022, δ = 0.21 | ||
Governance ratio | ||||
Sustainability ratio | p = 0.005, δ = −0.23 | |||
Ethics ratio | p = 0.034, δ = 0.14 | p = 0.011, δ = −0.14 | ||
Standards and codes ratio | p = 0.034, δ = −0.10 | |||
Total factors influenced | 7 | 2 | 5 | 2 |
Note: Empty cells indicate no significant correlation, and numbers represent the p-values of significant effects (p < 0.05) with corresponding Cliff’s delta (δ) measures of effect size; effect sizes are considered small when |δ| > 0.147, medium when |δ| > 0.33, and large when |δ| > 0.474
For example, the box toward the upper left of Table 2 showing a p-value of 0.004 is the medium-significance result of a Mann–Whitney U test comparing the concept map densities of students in the ME major versus those of EM/ISE students. Of the independent variables, academic major was a significant factor in the highest number of dependent variables (seven). While interaction effects were explored using Kruskal–Wallis H tests, the only significant results were in areas where there was already a known significant variable from the Mann–Whitney U tests. So, these interaction results are not presented or discussed.
4.1 Structural Analysis Findings.
The structural analysis revealed only one significant difference among the eight examined, specifically pointing out a medium difference in the concept map network densities of students in the ME versus the EM/ISE programs. Shown in Fig. 3, the EM/ISE students produced significantly denser concept maps than the ME students.
4.2 Thematic Analysis Findings.
Within the three major themes—engineering, business, and society—there were four significant findings from the analyses. The one with the largest effect size (δ = −0.49) is the influence of academic major on the business theme, illustrated in Fig. 4. As might be expected, students enrolled in the EM and ISE majors, which include management and operations topics in the curricula, consider business-related topics in their mental models of product design with higher frequency than students in ME majors.
On the other hand, ME students considered engineering terms with more frequency than EM/ISE students, seen in Fig. 5. As these three thematic ratios have a zero sum, this is where the ME students have shifted their focus; however, the effect size on this observation is small. Two different background factors had small significant influences on the society ratio in student concept maps: presence of a role model and intentions to pursue a master’s program. Students with engineering role models in their lives tended to include more society terms in their concept maps, see Fig. 6, as did students who were not pursuing a master’s degree, see Fig. 7.
4.3 Sub-thematic Analysis.
Because the sub-thematic level includes 12 categories, there are inherently fewer words per category than with the broader themes. There is also a wider array of usage with sub-themes. Some sub-themes were included often, while some were rare. For example, the average student included over 20% conceptual development terms in their maps, whereas the average student only included 0.2% governance terms. This is a likely explanation for why there were no statistically significant results in the latter category.
The remaining 11 significant Mann–Whitney U tests from Table 2 were associated with the sub-thematic response variables. Much like in the thematic analysis, students’ field of study influences several sub-thematic categories. The most significant are their ratios of prototyping and testing terms (see Fig. 8), market terms (see Fig. 9), and technical skills terms (see Fig. 10). Students majoring in ME used significantly more terms associated with prototyping and testing than those in the EM and ISE majors, with a medium effect size. ME students also used more technical skills terms than EM/ISE students, though it should be noted that the majority of students in both samples used no technical skills terms, as the median is zero; still, ME students were much more likely to include these terms. EM/ISE students, however, had substantially more market terms in their conceptual models. This can be viewed as a deeper dive into the trend from Fig. 4, as market is a sub-theme within the business theme.
Master’s program intentions had significant effects on students’ inclusion of sustainability and market terms, seen in Figs. 11 and 12. Those planning to pursue a master’s degree tended to include more market and fewer sustainability terms than those intending to seek an industry job after the bachelor’s degree. Interestingly, when examining the influence of student work experience, there was a significant effect on the inclusion of finance terms, shown in Fig. 13, where those with substantive work experience included fewer finance terms than those without. This effect is small, though, as the majority of students in both categories included no finance terms in their concept maps.
5 Discussion
The results provide insights into the primary research question posed at the beginning of this article: How do the backgrounds and academic profiles of third-year engineering students—specifically, their academic major and whether they have meaningful work experience, grew up with an engineering role model, and intend to pursue a master’s degree—influence their conceptions of product design, as measured through the breadth and thematic contents of individually generated concept maps? The statistical analyses revealed some differences among student conceptual models of design based on their academic major, intentions to pursue a master’s degree, meaningful work experience, and presence of an engineering role model in their lives.
5.1 Degree Programs and Design Conceptions.
The most significant of these factors, both in terms of frequency and effect size, is found to be the academic major. Students enrolled in the ME degree program included more terms associated with the engineering theme, particularly in the sub-themes of prototyping and testing and technical skills. The other group of students, who were enrolled in the EM and ISE programs, included denser concept maps with more business terms, particularly in the market sub-theme.
These differences across majors may be explained by the varying course curricula (prior to the Engineering Design VI course) between ME and EM/ISE students, as well as the predispositions of students who choose to pursue these major fields of study. By the time students reach their sixth academic term, EM and ISE students have taken courses such as project management, accounting and business analysis, and logistics and supply chain management, whereas ME students have taken courses such as fluid mechanics, design of machine components, and ME thermodynamics. When only looking at student majors, these course differences may explain the observed disparity between thematic and sub-thematic ratios. Additionally, the EM and ISE curricula include courses and content on complex systems and systems modeling, which may explain their tendancy to see more interconnections among their map elements (i.e., higher density). Another explanation for these findings is that students choosing to study ME are more inclined to focus on the technical engineering topics, whereas those choosing EM and ISE tend to think more about the broader system, including non-engineering factors.
These findings reinforce the observations of Silva et al. [21], who found that students in product development courses tend to focus their project work on particular areas associated with their fields of study. In their study, engineering students focused on engineering aspects like prototyping and technical development, and business students focused on marketability. Similarly, in the study reported here, ME students were found to think more about prototyping and technical skills, while EM students—who often consider themselves to be at the intersection of engineering and business—include relatively more market-related concepts.
Four small effects were found when comparing students who intend to pursue a master’s degree against those who do not; three of these effects are shown in Figs. 7, 11, and 12. Those who seek to continue in a master’s program included, on average, more market and manufacturing and production sub-theme terms and fewer society terms, particularly in the sub-theme of sustainability. One possible explanation is that students seeking to go into industry are more concerned about making societal and sustainability-related impacts, whereas those intending to continue their studies are more cognizant of complex business models and market factors, including marketing themselves for higher-level corporate positions.
5.2 Backgrounds and Design Conceptions.
Two of the independent variables studied—meaningful work experience and presence of an engineering role model—concerns students’ individual backgrounds and experiences. For each of these factors, only two of the 17 dependent variables were found to be affected, with small effect sizes at best. Regarding the presence of an engineering role model, those students who did have a role model tended to include more society terms and fewer manufacturing and production terms. Role models in an academic field, particularly in engineering, are known to shape a student’s sense of belonging, attitude, and motivation toward that field [17–19]. The results of the study reported here show specifically that such role models help students consider the context of society when thinking about product design. This may come at the expense of considering the specific downstream engineering concepts of manufacturing and production, though the effect size is smaller.
The second of these variables is whether a student has meaningful work experience in the form of an internship or co-op. Interestingly, students with such work experience were found to use fewer terms associated with finance and ethics. This lack of focus on non-technical aspects of design may be related to the type of siloed work that undergraduate engineering student interns are assigned. Further research is needed to dissect the students based on the type of work experience they performed in companies. The lack of significant results associated with this variable mirrors the results of Rulifson and Bielefeldt [25], who found mixed influences of internships on understanding of contextual factors.
5.3 Recommendations for Future Work.
The findings provide numerical evidence about the ways that students think about product design as they enter a sixth-semester engineering design course. In particular, they shed light on how background factors and academic degree programs may specifically influence the way students think about design, which allows course and curriculum designers to better understand how different student populations have different needs and gaps in their knowledge when beginning a course on holistic engineering design. As the results may only be interpreted within the context of one university and a relatively homogenous sample of students, it also provides a methodological framework for others to follow, both in the context of assessing prior knowledge of students in a classroom and in the context of engineering education research.
This article supports the need to further analyze students’ prior experiences, both within their academic programs and outside the classroom. One recommendation for instructors of future design courses is to collect data at the beginning of the course, and then tailor the course syllabus to the gaps in student conceptual models. This could be done in a comprehensive way through concept map collection and analysis, as was done in this study. However, this is time intensive, and so instructors may more easily survey their students about their backgrounds and infer learning needs from the correlations revealed in this and other similar studies. Such actions may lead to students who are better able to put their technical training into context, and institutions will build a stronger, more well-rounded pipeline of students who can approach design problems in holistic ways.
This research creates a foundation upon which further studies may build. The methodology and findings presented in this paper reveal several opportunities to supplement and expand on this domain. As the study took place in three programs at one private institution with a high proportion of white students, one direction is to expand to a more diverse population. Including students from different institutions and in different fields of study would yield more robust results that may provide additional support to generalize (or differentiate against) the findings in this paper. Furthermore, future research may include more depth in the demographic, background, and academic profile variables. In the study reported here, each independent variable was binary (e.g., yes or no, ME or EM/ISE). However, there are further details about the students which could be expanded into additional or more complex independent variables (e.g., type of work experience, education level of parents/guardians). While the sample size in the present study made this unlikely to produce statistically meaningful results, as the subsets of students would be quite small, a larger sample may make such a follow-up study more suitable. This would also open the door to include additional types of data that extend beyond the binary variables included in this article.
Lastly, an opportunity is presented to further refine the methods by which the concept maps are analyzed. Through more advanced network analysis strategies and/or concept map analysis tools, further research may uncover additional findings beyond the dependent variables utilized in this study. Two specific ideas are to investigate specific node pairings and to research trends in the ways certain themes connect with other themes within the concept maps. Furthermore, research to rigorously develop an industry-based expert concept map around the topic of product design could enable a dependent variable that measures concept map quality in a meaningful way, which can enable conclusions regarding which groups of students are better prepared for careers in industry.
6 Conclusion
This study took an exploratory approach to identify in what ways student backgrounds and academic degree programs may influence their conceptions of product design. Four academic and background factors were considered: academic major, intentions to pursue a master’s degree, internship or co-op experience, and the presence of an engineering role model in their lives. The most influential factor was academic major. Students in the ME program tended to consider more engineering factors such as prototyping, testing, and technical skills, whereas those in the EM and ISE programs considered more business-related aspects of design such as market factors. The EM and ISE students also had significantly denser concept maps, indicating a tendency to recognize more interconnections among the concepts they included in their concept maps. The other three factors—students’ intentions to pursue a master’s degree, whether they had an engineering role model, and whether they had meaningful work experience—were also indicators of small differences in the thematic contents of their concept maps. These findings provide insights on the gaps in students’ knowledge about holistic product design, the ways that outside factors and experiences may or may not be able to fill those gaps, and a baseline upon which educators can use to design improved engineering curricula for today’s students.
Acknowledgment
This material is based upon work supported by the U.S. National Science Foundation under Grant Numbers 1927037 and 1927114. Any opinions, findings, and conclusions or recommendations expressed in this material are those of the authors and do not necessarily reflect the views of the National Science Foundation. The authors gratefully acknowledge the support as well as the contributions of the study participants.
Conflict of Interest
There are no conflicts of interest.
Data Availability Statement
The datasets generated and supporting the findings of this article are obtainable from the corresponding author upon reasonable request.
Appendix: Survey Text and Response Options
Which of the following work experiences have you had in a technical role? (Check all that apply)
Summer internship, this past summer (2020)
Summer internship, previous summer (2019 or earlier)
Co-op
Co-op (select this if you’ve done more than 1 co-op)
Research with a faculty member at Stevens
Research at another institution
None of the above
Please list the companies that you have worked for in internship or co-op positions
What other (technical or non-technical) jobs have you held that do not fit the above categories? Please list the role and company
What was your primary role in your internship and co-op positions? (Check all that apply)
Project management or scheduling
Technical design
Non-technical design
Manufacturing
Logistics and supply chain management
Data analytics
Finance
I have not had internship or co-op experiences
Are you planning to complete a master’s degree at Stevens? (Mark only one)
Yes, I am in or considering the Accelerated Master’s Program (AMP) or 4+1 program
Yes, but not through the AMP or 4+1 program
Possibly
No
If you are completing or considering a master’s at Stevens, in what discipline will it be?
What courses are you currently taking (Spring 2021 term)? (Check all that apply)
(Included list of typical major-specific courses)
Which of the following courses have you already taken (BEFORE Spring 2021 term)?
(Included list of typical major-specific courses)
What is the highest level of education of your parents/guardians? (Choose the highest level among your parents/guardians)
No formal education
High school diploma or GED
College degree
Vocational training
Bachelor’s degree
Master’s degree
Professional degree
Doctorate degree
Unsure/prefer not to stay
Did you grow up with a parent, guardian, or close adult role model who has/had an engineering backgrounds? (Check all that apply)
Yes, at least one with an engineering degree
Yes, at least one with experience working as an engineer
Yes, at least one with engineering research experience
Yes, more than one with some engineering degree or work experience
Not sure
No
Your gender (Optional)
Female
Male
Non-binary
Prefer not to say
Other
Your age (Optional)