Abstract

As additive manufacturing (AM) usage increases, designers who wish to maximize AM’s potential must reconsider the traditional manufacturing (TM) axioms they may be more familiar with. While research has previously investigated the potential influences that can affect the designs produced in concept generation, little research has been done explicitly targeting the manufacturability of early-stage concepts and how previous experience and the presenting of priming content in manufacturing affect these concepts. The research in this paper addresses this gap in knowledge, specifically targeting differences in concept generation due to designer experience and presenting design for traditional manufacturing (DFTM) and design for additive manufacturing (DFAM) axioms. To understand how designers approach design creation early in the design process and investigate potential influential factors, participants in this study were asked to complete a design challenge centered on concept generation. Before this design challenge, a randomized subset of these participants received priming content on DFTM and DFAM considerations. These participants’ final designs were evaluated for both traditional manufacturability and additive manufacturability and compared against the final designs produced by participants who did not receive the priming content. Results show that students with low manufacturing experience levels create designs that are more naturally suited for TM. Additionally, as designers’ manufacturing experience levels increase, there is an increase in the number of designs more naturally suited for AM. This correlates with a higher self-reported use of DFAM axioms in the evaluation of these designs. These results suggest that students with high manufacturing experience levels rely on their previous experience when it comes to creating a design for either manufacturing process. Lastly, while the manufacturing priming content significantly influenced the traditional manufacturability of the designs, the priming content did not increase the number of self-reported design for manufacturing (DFM) axioms in the designs.

1 Introduction

Additive manufacturing (AM) has rapidly advanced to become a powerful tool for developing end-use products. AM processes can, in many cases, produce products faster than traditional manufacturing (TM), satisfying the customer’s needs swiftly [1]. With the desirable cost savings that AM can provide through benefits such as free complexity and mass customization, there is an incentive for designers to expand their approach to manufacturing to encompass AM technology in addition to TM technology, such as casting, injection molding, and machining. However, the lesser restrictions and expanded design freedom that are associated with design for AM (DFAM) encourage designers to rethink their current designs in favor of this new domain. In rethinking their designs, considerations must be made to account for the differences between design for TM (DFTM) and DFAM [2]. Broadly speaking, DFTM promotes the use of simple designs and minimizes the number of parts/components to produce the design as quickly and easily as possible. In contrast, DFAM encourages the use of intricate geometric designs and functional complexity without hindering the manufacturing time and the process of assembly.

While research has explored how early-stage design interventions can help encourage DFAM use, especially regarding manufacturability [3,4] and innovation [5], there is little research investigating the natural tendencies of designers, especially those still in their engineering education, to pursue either traditionally manufacturable or additively manufacturable designs during concept generation and the factors that can influence these tendencies. This is an important area to investigate because any concepts that are prematurely discarded or avoided in early-stage design due to their perceived manufacturing infeasibility [6] may be feasible when looking at all possible manufacturing methods to produce the design. Furthermore, there is a need to understanding if designers’ natural inclinations favor TM or AM early on in their design concept generation when manufacturing objectives in open-ended design prompts are not to be considered. If designers are only producing designs using the traditionally manufacturable design space, they must expand that design space to encompass the additively manufacturable design space to incorporate additively manufacturable solutions. As such, research is needed to determine if student designers’ early concepts are more naturally suited for TM or AM when manufacturing constraints are not introduced early in the design process, and how their previous experiences with manufacturing influence this underlying tendency while also observing if priming them with targeted manufacturing content can influence their tendencies. While the introduction of priming content to preoccupy one’s creative thinking toward the domain of interest has been implemented in diverse applications that include both academia [7,8] and industry [9,10], this new focus on both traditional processes and additive processes allows us to understand what student designers are inherently designing for and see whether their self-evaluation of the design’s manufacturability changes based on their previous experience and exposure to priming content.

2 Related Work

To properly contextualize the research in this paper, it is first important to understand the roles of expertise and content priming within concept generation (Sec. 2.1), and how DFAM challenges existing notions of manufacturability, especially in early-stage design (Sec. 2.2).

2.1 The Role of Expertise and Content Priming in Concept Generation.

Expertise is a significant factor that contributes to the types of designs that one may produce. Expertise can come from gaining experience in settings such as a job or in a classroom. Previous experience relating to a relevant domain can enable a designer to create novel ideas that novices without experience cannot [11]. Expertise can also impact the concept generation phase negatively, as designers with previous experience may prematurely discard ideas created in the brainstorming period based on intuition of infeasible solutions [12]. This practice is discouraged, as using the brainstorming period to create as many designs as possible yields greater creativity and better performance [13]. The ideas produced in a concept generation phase can be impacted by many factors, with the designers’ previous experience representing a strong influence [14]. These factors may result in experienced designers developing advanced ideas while also discarding potentially viable concepts.

In contrast to experienced designers, novices may lack the previous experience that aids in the decision-making process. Cross [15] extensively studied the differences between experts and novices, where it was found that novices had difficulty setting up and defining the problem statement and would have their cognitive activity decline in working through an experiment. Ahmed et al. [16] found that novices expressed uncertainty in their design decisions, resorting to using trial-and-error methods as opposed to strategizing early in the design process, and expressed difficulty working with unfamiliar task. The work by Cross and Ahmed indicates there is a need to provide tools to novice designers to help them develop designs early in the concept generation stage. By providing students with tools, such as priming them with relevant content to help influence their design decision-making, they can develop designs that reflect those created by experts. Such priming may take the form of design heuristics, such as geometry modification, design flexibility, and functional adaptivity [17], meant to stimulate the creation of high-quality designs.

Priming content can enable designers to solve a design problem based on the material that is provided to them. During the priming process, the discussed content is brought to the forefront of the designers’ minds. This priming process enables people to retrieve older information while retaining any newly introduced information, as was found by Ratcliff and McKoon [18], Tulving and Schater [19], and Schacter and Buckner [20]. Likewise, Bonnardel and Marmeche [21] showed from their testing that it is possible to prime designers with specific content. Additionally, how receptive the designers were to the priming process was found to vary based on their individual previous experience. Priming designers prior to concept generation activities can also help align their designs to better reflect the priming content. For example, Yilmaz et al. [22] found that priming designers with design heuristics early in the design process is an effective way to produce creative and diverse designs. In the context of AM, Atatreh et al. [23] found that in accelerated learning environments it is feasible to effectively deliver AM content to students through a means that they can understand the new information. In addition, Lauff et al. [24] found that when priming students with AM design heuristic content, students create designs of higher novelty. By incorporating design for manufacturing (DFM)-related priming into design work, the participants can choose what they want to incorporate into their designs, rather than subconsciously forgetting about them when they need to be recalled.

2.2 Consideration of Design for Additive Manufacturing Within Concept Generation.

Due to the distinct differences between DFTM and DFAM concepts, significant research is being performed toward generating content appropriate to act as priming in design activities. As an example, Laverne et al. [25] developed priming content for AM’s design considerations, which can be categorized into three groups: opportunistic-DFAM (O-DFAM), restrictive-DFAM (R-DFAM), and dual-DFAM. O-DFAM refers to the capabilities of AM, such as geometric complexity and topology optimization, while R-DFAM refers to the limitations of AM, such as material selection and machine constraints. Dual-DFAM involves the inclusion of both O-DFAM and R-DFAM in the design process. These categories of DFAM design considerations enable us to better evaluate designs for AM, as evidenced by the AM design principles established by Perez et al. [26].

By introducing these DFAM design considerations in the form of manufacturing priming content, researchers are exploring how such content impacts the outcomes of design activities, especially in early-stage design. To minimize the wasted costs and time that occur as a result of redesigning later in the design process, it is important to encourage full exploration of possible design considerations early in the design process [27]; this is especially important as AM continues to expand the viability of more complex designed geometries. Design considerations are often best presented in the form of design heuristics, which compose the key features and functional elements that make up a design [28]. Blösch-Paidosh and Shea [29] found that students’ designs are influenced when presented with design heuristics relevant to AM. While their methodology of introducing DFAM heuristics cards [30] was found to improve the designs that students create in favor of AM [31], the changes were not radical, as verified by the work done by Yang et al. [5]. This concept of rethinking designs through the introduction of AM heuristics was further explored by Watschke et al. [32]. While the authors found that introducing AM knowledge to students and designers in the early stages of the design process can enable even non-experts of AM to consider incorporating AM heuristics into their designs, they acknowledge that for utilizing the full potential of AM, students and designers also need to have both design experience and a more in-depth understanding of AM.

Regarding the implementation of O-DFAM and R-DFAM, although O-DFAM was found to be better by itself in the early stages of the design process compared to R-DFAM [33], prior research has shown that students who are primed with only O-DFAM or R-DFAM result in framing students to create designs that are not fully realized for AM [3]. To create designs that are ideal for AM, students must be introduced to both O-DFAM and R-DFAM together in a dual-DFAM format. To test the effects of AM priming on students’ designs, Prabhu et al. [3,4,34,35], developed manufacturing priming content based on Laverne’s categorization and studied its effects on undergraduate students’ concept generation. In their experiments, three groups of students were given a design challenge along with either (1) no-DFAM priming content, (2) only R-DFAM, or (3) dual-DFAM. The results showed that students primed with only R-DFAM emphasized a design objective of minimizing build time, while students primed with dual-DFAM emphasized a design objective of minimizing build material [34]. Additionally, designs generated by the R-DFAM group incorporate more appropriate tolerances with easily accessible support material but also tend to have higher build plate contact area when compared with designs from the dual-DFAM priming [35]. Further, participants from all three groups reported higher use of R-DFAM axioms, compared with O-DFAM axioms [5].

While Prabhu et al. researched the effects of AM priming on the additive manufacturability of designs [3], results do not account for the design’s manufacturability through TM and along with associated DFTM considerations [36]. There is a need to better understand the manufacturability of the designs in the context beyond just AM, as Seepersad [37] identified the need for designers to break free from their TM mindset when creating designs for AM. While some of these designs reflect the design considerations associated with AM, an evaluation for TM has not yet been done. A lack of in-depth understanding of how 3D printing affects design thinking has been acknowledged as an area of interest [38], but has yet to be further investigated. These design considerations, which often emphasize the simplicity of designs, may be subconsciously present given the designer’s previous experience and interpretation of any provided priming content. Currently, there is a lack of research on the manufacturability of designs when evaluating for TM and AM, which this paper investigates. By evaluating participants’ generated designs in terms of manufacturability for both TM and AM with manufacturing priming content to invigorate their minds for creative thinking, the effects of previous manufacturing experience and content priming can be better assessed. Furthermore, by assessing the participants’ designs early in the design process (i.e., before they would be asked to consciously select a manufacturing approach), the natural inclinations of designers’ toward AM or TM can be identified.

3 Research Objectives

Considering the existing body of research, the objectives of this paper are to determine whether students’ previous experiences with TM, AM, and/or the introduction of manufacturing priming content influence (1) the manufacturability of the designs with regard to TM and AM and (2) their self-reported use of different DFTM and DFAM axioms in their designs. Through this investigation, this work will understand how experience and manufacturing priming content affect whether designs are inherently more suitable for TM (typically geometrically simple) or AM (typically geometrically complex). The scope of this research focuses on students in academia, where future work will expand further toward investigating the effects seen in professionals working in industry. The following research questions are proposed:

  1. What effects do content priming and experience have on expert-assessed manufacturability?

    H1.1: We hypothesize that designs from students at a low self-reported manufacturing experience level will tend toward TM. Participants at this experience level are those who would respond in the range of a 1–2 on a five-point Likert Scale, which will be discussed in Sec. 4. This hypothesis assumes that all the participants in this study, regardless of their self-reported formal DFM experience levels, likely have extensive informal experience with TM. With many manufacturing businesses in the US leveraging traditional processes [39], along with growing efforts toward incorporating manufacturing into university curriculum [40], this current dominance in technological manufacturing leads to more products today being produced using TM. The everyday exposure to products made from TM may cause the participants to be informally trained in the design considerations used to make them.

    H1.2: As manufacturing experience increases in either type of manufacturing (TM or AM), we anticipate the resulting designs to likewise increase in that type of manufacturability (i.e., higher experience in DFAM will lead to designs that are more suitable for AM). This is based on the notion that designers will rely heavily on their previous experience when creating new designs [41].

    H1.3: Regarding the priming content, we hypothesize that students who receive the manufacturing priming content (for both TM and AM) will create designs that are better suited for manufacturing through either TM or AM. The intent of the manufacturing priming content is to bring existing DFM axioms to the forefront of the students’ minds. Through this process, the axioms are therefore more accessible to the students, which increases the likelihood of the students integrating the DFM axioms into ongoing perceptions, judgments, and choices regarding the design challenge [42]. While the students’ design intentions will not be collected as part of this study, it is anticipated that they will either leverage the DFTM priming content to create designs that are suitable for TM, or they will leverage the DFAM priming content to create designs that are suitable for AM.

  2. What effects do content priming and experience have on how the self-reported axioms are applied?

    H2.1: We hypothesize that students at a low self-reported manufacturing experience level will minimally apply DFM axioms in their designs. With applicability, usage, and familiarity of DFM axioms being linked to one’s previous experience with them [43], by being unfamiliar with these axioms, the students are unlikely to recognize and identify them in a critical assessment of their designs with confidence and accuracy. As manufacturing experience increases with either TM or AM, these students are likely more familiar and confident with the DFTM and DFAM axioms, respectfully. Therefore, they likely can more greatly recognize and apply axioms in their designs through self-assessment.

    H2.2: Regarding the priming content, we hypothesize that students who receive the manufacturing priming content will more greatly apply DFM axioms in their designs. By making the students aware of these DFM axioms and what they entail in an environment analogous to a workshop [44], they will be able to apply these axioms toward evaluating the manufacturability of their designs, resulting in higher levels of self-reporting. While factors such as content retention [45] and the order in which the information is presented in Refs. [4648] limit conclusions being made regarding which axioms will specifically be reported, it is anticipated that presenting the DFM axioms will increase recognition and applicability in a self-assessment process.

4 Methodology

To answer the research questions, an experiment was developed to test the effects of previous manufacturing experience and manufacturing priming on students’ generated designs. The experiment consisted of three stages: (1) a pre-intervention survey, (2) manufacturing priming for a select group of students, and (3) a design challenge followed by students’ self-evaluations of their designs. The study was reviewed and approved by the Institutional Review Board, and implied consent was obtained from the participants prior to the experimentation. In this experiment, the participants first reported their current level of expertise with TM and AM. Next, depending on their assigned experimental group, students would receive priming content for both manufacturing technologies to bring these concepts to the forefront of their minds. From there, they were asked to complete an open-ended, manufacturing-agnostic design challenge. They then completed the experiment by self-evaluating their designs for TM and AM based on the axioms presented in the initial content priming. Finally, after the design activity, participant designs were evaluated by manufacturing domain experts. The following subsections discuss the further details behind experimentation and analysis.

4.1 Participants.

Participants were recruited from several upper-level undergraduate engineering design courses (113 students in a 3rd-year class and 82 students in a 4th-year class) and one graduate engineering design course (48 graduate students) at a large northeastern university. The experiment was performed over the course of two semesters, where the participants in the Fall semester (n = 95) received the manufacturing priming content. In contrast, the students in the Spring semester (n = 148) did not receive any manufacturing priming content prior to the design activity; this group of participants received the manufacturing priming content after the study to ensure all students understood the intent of the study and its relevance to their DFM curriculum. Within the respective semesters, the experiment took place at a time when the manufacturing priming content would be relevant to their ongoing coursework, thereby encouraging the students to participate as it would be naturally beneficial for them. Some participants’ data (not included in these listed n-values) were removed from consideration due to incompleteness in the activity where key information was critical (i.e., missing the self-reported evaluation for “considering the manufacturing size”) or the key information was not filled in properly (i.e., the self-reported evaluation for “avoiding large, flat regions” had two scores filled in when only one was requested).

4.2 Procedure and Metrics.

The following section outlines the experiment that was conducted with the primed and unprimed students and how the findings were analyzed. This includes the initial survey used to gather information on the students, an overview of the manufacturing priming content used for a select group of participants, the ensuing design challenge, and the involvement of expert evaluators in assessing the designs created for the design challenge.

4.2.1 Pre-Intervention Survey.

At the outset of the activity, participants were given 5 min to complete a survey that asked about their demographics (specifically, their gender, age, year of study, and undergraduate major) and their previous experience with TM and AM separately on a five-point Likert-type scale [49], with a score of 1 representing “I have never heard about TM/AM before this” and a score of 5 representing “I am an expert on TM/AM and can proficiently manufacture parts.” Likewise, they were also asked to evaluate their familiarity with a series of 14 different DFTM and DFAM axioms (seven for each) on a five-point Likert-type scale [49], with a score of 1 representing “Never heard about it” and a score of 5 representing “Could regularly integrate it with my design process.” This survey, which was modified from the studies done by Prabhu et al. [4,34], provides the research team with an understanding of participants’ current levels of TM and AM experience and familiarity with the respective DFM axioms. A link to the survey (and the experimental packet given to the students) can be found on the research team’s website [50]. Because the collection of the students’ expertise with manufacturing is solely self-reported, the students may over or underestimate their manufacturing experience, as students have demonstrated in prior research to misinterpret their manufacturing experience [51]. While the proposed Likert self-efficacy scale has been verified for providing one’s self-efficacy with AM based on their prior experience with AM and self-reported familiarity with DFAM axioms [52], the validity of one’s self-efficacy with TM based on their prior experience with TM and self-reported familiarity with DFTM axioms cannot be confirmed in this study. While the TM and DFTM Likert scales were created by incorporating the axioms [53] into a survey tool that strongly resembles the verified scale from Prabhu et al. [52], it is recognized that the students may provide an inaccurate assessment of their self-efficacy with TM and AM that can lead to overlapping between the manufacturing experience levels.

4.2.2 Manufacturing Priming.

After completing the pre-intervention survey, the students in the Fall 2021 semester received a 20-min lecture that communicated the priming content on DFM, encompassing both traditional processes and additive processes. The purpose of the manufacturing priming content was to not teach the students about DFM but rather to provide information that they may be aware of before proceeding to the design challenge; the students were not told about a design challenge until after the manufacturing priming content had concluded. Furthermore, the intent of the priming content was to not skew the designs that the students would create to become suitable for one manufacturing process over another but rather to bring the concepts to the forefront of their mind in an unbiased manner; the purpose of introducing this lecture is to understand how the students’ designs would change regarding their manufacturability based on the exposure to priming content. A similar approach for priming designers with the intent of influencing their idea creations was conducted by Liao et al. [10]. Due to the wide range of available TM processes, casting, injection molding, machining, and sheet metal forming were used as representative TM processes. Despite the initial impressions that these individual TM processes differ from one another, the design considerations that must be factored in when using the respective processes overlap each other [53]. These similarities resulted in the generalized grouping in the set of TM processes. Similarly, the design considerations for the individual AM processes also have overlap with each other, hence why we are able to provide general DFAM axioms that represent the selected processes of material extrusion, powder bed fusion, vat photopolymerization, and material jetting.

The priming content began by defining the concept of designing for manufacturing, which was followed by a brief overview of the two types of DFM axioms (DFTM and DFAM) relevant to this research. The ensuing priming content discussed fundamental design considerations in the form of axioms for both manufacturing processes. To keep the experiment balanced, seven axioms were identified for each process (resulting in 14 manufacturing axioms to discuss). Each axiom was presented on its own separate slide, with a one-sentence summary and visual content comprising each slide. The time given to discuss both sets of axioms was balanced. Participants in the Spring 2022 received the manufacturing priming content after completing the post-intervention survey. This was done to ensure that all participants across both semesters received the same information and came away from the activity with a better understanding of DFM.

For the experimental group that received the manufacturing priming content, TM was first introduced, where the following principles [53] were discussed: (1) reducing part count, (2) relying on low-labor-cost operations, (3) avoiding intricate shapes, (4) utilizing standard materials, components, and tooling, (5) avoiding sharp corners by using fillets, (6) using a uniform wall thickness, and (7) having ample spacing between holes. Next, AM was introduced, where the following principles [35] were discussed: (1) incorporating complex shapes and geometries, (2) combining multiple parts into a single part or assembly, (3) avoiding large, flat regions, (4) orienting overhanging surfaces, (5) considering the minimum feature size, (6) orienting curved surfaces, and (7) accounting for potential variations in material properties. These principles, or axioms, provide broad guidelines that are applicable to the many TM and AM processes that exist. Examples of the content used for the manufacturing priming for TM and AM are shown in Figs. 1 and 2, respectively.

Fig. 1
Sample TM priming content
Fig. 1
Sample TM priming content
Close modal
Fig. 2
Sample AM priming content
Fig. 2
Sample AM priming content
Close modal

4.2.3 Design Challenge and Procedure.

Following the content priming, students were given the design prompt that they would be solving. To avoid overwhelming the students with a functionally complex design prompt to be solved in a short duration of time, the purpose of the design challenge was to provide the students the opportunity to design a solution for a simple problem where the utilized design space could equally account for TM and AM without any fixation. In identifying a design problem that aligns with this study’s purpose, which is to introduce a process-agnostic design challenge that can allow the students to freely explore the design space based on their natural inclinations, the chosen design challenge came from Prabhu et al. [54]. This design challenge consists of two components: a visual guide (which is shown in Fig. 3) and the following design prompt: “You are tasked with designing a solution to hold three hollow tubes securely in place and parallel to each other. All tubes must be held 2 in. away from a fixed wall (measuring from the wall to the closest edge of the tubes). The tubes are 1 in. in diameter and 3 in. long.” This design challenge in particular is applicable to the students as it falls in line with the shift toward problem-based learning [55] observed in academia, which improves students’ ability to understand and retain the newly introduced concepts and knowledge [56]. To remove any potential manufacturing biases in the design challenge, students did not receive any manufacturing constraints in the design prompt itself. By removing any manufacturing-specific constraints from the design challenge, such as making the design as light as possible or having maximum strength, the students can use their design space to create any solution that meets the design prompt’s criteria. In this way, the design task does not necessarily require or benefit from AM or TM designs, putting them on equal footing during the concept generation task.

Fig. 3
Design challenge visual provided to participants
Fig. 3
Design challenge visual provided to participants
Close modal

After reading through the design challenge prompt, students spent 10 min using the provided design sheets to individually create as many solutions as possible. They were instructed to use both sketches as well as text to illustrate their designed solutions. While the students were creating designs in the concept generation session, they were also asked to describe the advantages and disadvantages of each design concept. An example of a completed design sheet is shown in Fig. 4.

Fig. 4
Example of completed design sheet
Fig. 4
Example of completed design sheet
Close modal

Following the concept generation session, participants were given 7 min to identify a final design. They were informed that their final design could be any of the following: a reused or modified design from the initial concept generation period, a combination of any of the previous designs, or an entirely new idea. As with the initial concept generation session, participants were asked to list the advantages and disadvantages of their final design. After identifying their final design and discussing its strengths and weaknesses, participants were asked to self-evaluate their solution as designed based on the seven DFTM axioms and seven DFAM axioms they were asked about in the pre-intervention survey to the best of their ability. Specifically, participants were presented with each axiom and asked, “To what extent do you agree with the following statements about manufacturing as they apply to your final design?” They then evaluated the design using a five-point Likert ordinal scale, where 1 represented “Strongly Disagree” and 5 represented “Strongly Agree.” While some research considers Likert scales to be interval [49], the Likert scale was originally developed with ordinal properties [57] and will be used as such for the analysis in this work. While the spacing between options such as “Strongly Disagree” and “Disagree” may be unequal, the hierarchal ranking of the five options means that the self-reported assessment of the designs can be analyzed for common trends across the Likert scale. This self-evaluation allows the researchers to observe which manufacturing priming content principles (if any) influenced students’ designs.

4.3 Expert Design Evaluation.

To evaluate participants’ final designs, three raters (two experts and one quasi-expert in design for manufacturing processes) used the consensual assessment technique (CAT) as developed by Amabile [58]. This technique has expert judges evaluate creativity in their specialty domain [59]. Pertinent to the research in this paper, the CAT has also previously been used to evaluate the suitability of design concepts for manufacturing [54,60]. Both expert raters have graduate degrees, have at least six years of experience with creating and evaluating designs for AM, and previously published papers in the relevant field. The quasi-expert is currently progressing through graduate coursework and has experience with creating and evaluating designs for AM. The three raters evaluated the final designs based on both their traditional manufacturability (TM CAT) and additive manufacturability (AM CAT). Both categories of manufacturability were evaluated on a 1–6 scale, with higher scores indicating greater suitability for that manufacturing process type. In evaluating the designs for the simple task, the raters were instructed to evaluate the designs for TM and AM separately, where multiple TM processes and multiple AM processes could be used to manufacture the design, but they could not be mixed (more specifically, hybrid manufacturing [61] was not used in assessing the designs). A brief description of each category is as follows:

  • Traditional manufacturability (TM CAT): The suitability of the design for TM based on expert assessment. Though a variety of traditional processes are possible, scoring is based on the assessment of applicable DFM principles. A higher score represents a design that utilizes the general principles of TM (simple shapes, rounded corners, ample spacing between holes, etc.) while a lower score represents a design that is either very difficult or impossible to manufacture using TM processes.

  • Additive manufacturability (AM CAT): The suitability of the design for AM based on expert assessment. The category here focuses on the use of both R-DFAM and O-DFAM principles in the design and how they apply to the various AM processes. A higher score represents the use of most R-DFAM and O-DFAM principles, while a lower score represents little to no identifiable R-DFAM and O-DFAM principles. Intermediate scores tend to exhibit suitable R-DFAM, but lack in O-DFAM.

The three raters first jointly scored ten randomly selected submitted designs from the Fall 2021 semester to establish the evaluation criteria and achieve general agreement. Next, each rater individually scored a set of 40 randomly selected designs from the Fall 2021 semester which were then compared for consistency. The scores were validated for consistency using the interclass coefficient (ICC) [6264]. The ICC value was calculated using spss v.29, which yielded a strong general agreement with a Cronbach’s α of 0.786 for the traditional manufacturability rating and an α of 0.782 for the additive manufacturability rating, both of which exceed the minimum threshold for meaningful agreement of 0.75 [65]. These α values were also significant with a p-value of <0.001 using a 95% confidence interval. This means that for each design the raters were giving comparable scores. From there, the raters scored the remaining 45 designs from the Fall 2021 individually. For the Spring 2022 semester, the raters scored all designs individually based on previous evaluation criteria and ICC values were calculated for overall reliability. The cumulative α values across the three raters for traditional manufacturability were 0.835 and the additive manufacturability rating was 0.795, both of which were significant with a p-value of <0.001 using a 95% confidence interval. This indicates a good agreement between the raters for all experimental conditions.

After the raters evaluated all the designs, the average TM CAT score and average AM CAT score were calculated for each student by averaging the respective CAT scores provided by the raters, where a higher score indicates that the design is suited for its respective manufacturing process. Examples of designs that received a high TM CAT score and a high AM CAT score are shown in Figs. 5 and 6, respectively. The expert raters interpreted the design shown in Fig. 5 as three C-clamps attached to the wall with screws. In contrast, the raters depicted the design shown in Fig. 6 as a block attached to the wall that is infused with lattice structures.

Fig. 5
Design example with high TM score
Fig. 5
Design example with high TM score
Close modal
Fig. 6
Design example with high AM score
Fig. 6
Design example with high AM score
Close modal

5 Results

To communicate key data collected through this study, this section first details the distribution of students’ manufacturing experience (Sec. 5.1), followed by statistical analysis using spss v.29 to answer the research questions posed for expert evaluation of manufacturability (Sec. 5.2) and self-reported use of DFM axioms (Sec. 5.3).

5.1 Distribution of Student Manufacturing Experience.

Before analyzing the manufacturability of the participants’ designs, it is first necessary to observe the distribution of the participants’ experience with manufacturing to get an understanding for their overall individual manufacturing experience level. Figure 7 shows the students’ manufacturing experience distribution for both TM and AM processes. The student distribution shows that at each experience level, there was approximately the same number of students with the requisite TM experience and AM experience. Most participants claimed an experience level between 2 and 3 for both TM and AM processes, which was expected based on the assumption that the manufacturing experience of the students would be normally distributed.

Fig. 7
Manufacturing experience distribution
Fig. 7
Manufacturing experience distribution
Close modal

The results in Fig. 7 appear to indicate that the small sample sizes for experience levels 1 and 5 will make conclusions about these populations difficult. These small sample sizes are likely attributed to the target recruitment of the participants that come from the upper-level engineering courses, where the majority of these students neither have the manufacturing experience of professionals nor the manufacturing inexperience of novices [66]. While conclusions will be made about the differences between students across the entire spectrum of the manufacturing experience levels (i.e., comparing students with low manufacturing experience against students with high manufacturing experience, as will be discussed shortly) conclusions made about the differences between students based on specific manufacturing experience levels (i.e., comparing students with an AM experience level of 1 against students with an AM experience level of 2) may change as more data is collected at the lowest and highest levels of manufacturing experience.

The similarity in experience scores between both TM and AM processes prompted an additional analysis to see if there was a correlation between a participant’s TM experience and AM experience. Figure 8 collects the paired experience scores for each individual participant. This figure suggests that there is an interconnectedness between a participant’s previous experience with TM and their previous experience with AM. This is likely the result of students having equal exposure to both types of manufacturing processes in their academic studies based on the assumption that students who are learning about manufacturing are equally exposed to TM and AM.

Fig. 8
Manufacturing experience comparison
Fig. 8
Manufacturing experience comparison
Close modal

To verify the presence of a significant relationship in the interconnectedness for participants’ manufacturing experiences between TM and AM, Spearman’s rank-order correlation test was performed. The test yielded an R-value of 0.510, signifying a moderate correlation [67] between a person’s TM experience level and AM experience level. Using a 95% confidence interval, this claim was verified by the statistically significant p-value of <0.001. This means that a student with a high TM experience level is likely to also have a high AM experience level. The implication of this correlation allows for further categorization by grouping students with low manufacturing experience together and students with high manufacturing experience together. More specifically, a student with either a low TM or AM manufacturing experience can be considered a student with a low overall manufacturing experience Similarly, a student with either a high TM or AM manufacturing experience can be considered a student with a high overall manufacturing experience. The significance of this correlation and its effect on the rest of the findings will be discussed in Sec. 6. While there is likely interconnectedness for the students’ manufacturing experience between TM and AM, because the correlation was only moderate, the following results will isolate the students’ manufacturing experience based on TM and AM rather than present the results using a conjoined manufacturing experience scale.

5.2 Expert Evaluation of Design Manufacturability.

To answer the first research question of this study, participants’ manufacturing experiences in each process, as collected earlier in Fig. 7, were initially used as the basis to compare the difference between TM CAT scores and AM CAT scores while distinguishing between the students that were primed/unprimed. These results are presented in Fig. 9 for TM experience and AM experience, respectively. As a reminder, a higher CAT value denotes designs that are more suitable for the associated manufacturing process.

Fig. 9
CAT score versus TM experience (left) and AM experience (right)
Fig. 9
CAT score versus TM experience (left) and AM experience (right)
Close modal

The averages and standard deviations for the CAT scores across both processes separated by experience level are shown in Table 1 for traditional manufacturability and additive manufacturability, respectively.

Table 1

TM CAT score average and standard deviation; and AM CAT score average and standard deviation

TM experience levelTM CAT score average (unprimed)TM CAT score standard deviation (unprimed)TM CAT score average (primed)TM CAT score standard deviation (primed)
14.5270.4383.5560.480
24.4860.3263.9430.318
34.1790.3164.0970.322
44.3620.2953.5750.447
52.6650.7202.8870.678
AM experience levelAM CAT score average (unprimed)AM CAT score standard deviation (unprimed)AM CAT score average (primed)AM CAT score standard deviation (primed)
12.8350.5772.8620.430
23.1280.1952.7530.286
33.4920.1943.0550.269
43.2660.2333.5510.325
54.0000.4084.3335.544
TM experience levelTM CAT score average (unprimed)TM CAT score standard deviation (unprimed)TM CAT score average (primed)TM CAT score standard deviation (primed)
14.5270.4383.5560.480
24.4860.3263.9430.318
34.1790.3164.0970.322
44.3620.2953.5750.447
52.6650.7202.8870.678
AM experience levelAM CAT score average (unprimed)AM CAT score standard deviation (unprimed)AM CAT score average (primed)AM CAT score standard deviation (primed)
12.8350.5772.8620.430
23.1280.1952.7530.286
33.4920.1943.0550.269
43.2660.2333.5510.325
54.0000.4084.3335.544

The plots in Fig. 9 show different trends in the CAT score for TM and AM as the experience level increases. Regarding traditional manufacturability, the average CAT score remains generally consistent as the TM experience level increases (the outliers at the lowest and highest experience levels are likely attributed to the relatively small sample size in each of these groups). In contrast, a slight increase in CAT score is seen for the additive manufacturability plot as the AM experience level increases.

It can be seen from the data in Table 1 that the standard deviation variations are much higher for TM CAT and AM CAT at the lowest and highest levels of self-reported manufacturing experience. As previously discussed, this high variability is likely due to the small sample sizes at these manufacturing experience levels. This further highlights the need to collect additional data at these experience levels, as conclusions made across specific manufacturing experience levels (i.e., AM experience level 1 compared to AM experience level 2) may change as the sample sizes in these manufacturing experience levels increase.

To test for statistical significance, two separate three-way analysis of variance (ANOVA) tests were performed to compare the results across manufacturing experience levels while accounting for the effects of priming. The manufacturing experience in each respective process (TM and AM) and the condition of primed/unprimed were set as the independent variables, while the CAT score for TM and AM was set as the dependent variable. The experience level data was treated as ordinal due to the clear separation and ranking between levels. Additionally, the CAT values were treated as interval because there was significant meaning in manufacturability scores in-between the raters’ evaluation (more specifically, a CAT score of 2.5 was possible and important). A 95% confidence interval was used to determine statistical significance (i.e., p < 0.05). The ANOVA test was chosen because all the CAT scores were normally distributed as assessed by the Shapiro–Wilk test [68]. These results are presented in Table 2 for the main factors and their interactions, respectively.

Table 2

CAT score statistical significance for main factors and interaction effects

CaseTM CAT P-valueTM CAT F-valueAM CAT P-valueAM CAT F-value
Primed/Unprimed0.033**4.6290.8080.059
TM XP0.3981.0190.7380.497
AM XP0.3001.2270.1151.879
CaseTM CAT P-valueTM CAT F-valueAM CAT P-valueAM CAT F-value
Primed/Unprimed0.033**4.6290.8080.059
TM XP0.3981.0190.7380.497
AM XP0.3001.2270.1151.879
Interaction effectsTM CAT P-valueTM CAT F-valueAM CAT P-valueAM CAT F-value
Primed/Unprimed and TM XP0.9050.2570.3931.030
Primed/Unprimed and AM XP0.3351.1480.8580.329
AM XP and TM XP0.9310.4510.4461.001
Primed/Unprimed, TM XP, and AM XP0.9220.3280.1861.481
Interaction effectsTM CAT P-valueTM CAT F-valueAM CAT P-valueAM CAT F-value
Primed/Unprimed and TM XP0.9050.2570.3931.030
Primed/Unprimed and AM XP0.3351.1480.8580.329
AM XP and TM XP0.9310.4510.4461.001
Primed/Unprimed, TM XP, and AM XP0.9220.3280.1861.481

Note: **p < 0.05.

The results in Table 2 show that only the TM CAT score was found to be significantly influenced by the manufacturing priming content when observed at the 95% confidence interval. These results appear to indicate that priming is only beneficial for those students with experience, which may seem to initially conflict with the findings in Fig. 9. To identify where this significant difference is occurring within the TM CAT data based on the priming content and TM experience, a series of paired t-tests were performed to evaluate the traditional manufacturability across each of the TM experience levels while separating the students who received the manufacturing priming content from those who did not. A 95% confidence interval was used to determine statistical significance (i.e., p < 0.05). The results for the manufacturability scores based on the priming content across each of the TM experience levels are shown in Table 3.

Table 3

Statistical significance for traditional manufacturability based on priming content based on TM experience

TM experience levelTwo-sided P-valueCohen’s d
10.1140.780
20.2620.182
30.6840.074
40.1590.370
50.746−0.214
TM experience levelTwo-sided P-valueCohen’s d
10.1140.780
20.2620.182
30.6840.074
40.1590.370
50.746−0.214

The results from Table 3 show that despite general significance for the traditional manufacturability scores based on the priming content, there were no significant differences observed across each of the experience levels. This means that students were significantly influenced based on the manufacturing priming content across the entire sample size, regardless of their self-reported manufacturing experience level. The implications of the coupling between manufacturing content priming and traditional manufacturability are discussed in Sec. 6.3.

Additional correlation tests were produced to compare the effects of manufacturing experience on the expert-assessed manufacturability of the designs. For this analysis, Spearman’s rank-order correlation test was performed. The data were analyzed using spss v.29 and a 95% confidence interval was used to determine statistical significance (i.e., p < 0.05). The correlation results are shown in Table 4.

Table 4

Correlations between manufacturing experience and manufacturability

Manufacturing experience typeManufacturability typeSpearman’s ρTwo-tailed P-value
TMTraditional−0.0870.176
Additive0.1390.031**
AMTraditional−0.1440.025**
Additive0.242<0.001**
Manufacturing experience typeManufacturability typeSpearman’s ρTwo-tailed P-value
TMTraditional−0.0870.176
Additive0.1390.031**
AMTraditional−0.1440.025**
Additive0.242<0.001**

Note: **p < 0.05.

The results in Table 4 show that the correlation between manufacturing experience and manufacturability is weak. Despite this lack of strong correlation, the correlation coefficient was higher for additive manufacturability than traditional manufacturability. This finding, which coincides with a significant p-value when evaluated with a 95% confidence interval, indicates that experience is more correlated with additive manufacturability than traditional manufacturability.

5.3 Self-Reported Use of Design for Manufacturing Axioms.

To answer the second research question, the participants’ evaluation sheets were categorized based on their identified experience levels with both TM and AM. Next, the data were recorded for what Likert score they assigned to each of the 14 DFM axioms presented in the priming content. To test statistical significance, 14 separate three-way ANOVA tests were performed on the data, which compared the independent variables (manufacturing experience and priming) to the dependent variables (self-reported score for each axiom). A 95% confidence interval was used to determine statistical significance (i.e., p < 0.05). This test was chosen in part due to the normally distributed self-reported scores for each of the individual DFM axioms as assessed by the Shapiro–Wilk test. Additionally, while the Likert scale for the self-reported scores was treated as ordinal [57], there is a loss in information when using non-parametric tests on Likert data [69]. With prior research in similar applications demonstrating that parametric and non-parametric tests yield similar trends across Likert data [70], it was determined that the ANOVA test would be sufficient for this analysis. The significant results are highlighted in Tables 5 and 6 for DFTM and DFAM axioms, respectively. For brevity, effects that did not demonstrate statistical significance are omitted here. Additionally, instances where statistical significance was observed with a 90% confidence interval (i.e., p < 0.1) were highlighted. While these cases may not be as significant to the same degree as the cases where significance was observed with a 95% confidence interval, based on the collected data it felt important to acknowledge these results as these relationships may become more or less significant (and therefore, more or less important) with additional data at the lowest and highest levels of manufacturing experience.

Table 5

Statistically significant DFTM axioms

DFTM axiomMain effect or interactionP-valueF-value
Reduce part countTM XP*AM XP0.098*1.610
Low-labor-cost operationsPriming0.080*3.100
TM XP*AM XP0.078*1.689
Standard materials, components, and toolingTM XP0.014**3.208
Priming*TM XP*AM XP0.019**2.591
Uniform wall thicknessAM XP0.071*2.196
TM XP*AM XP0.099*1.604
Ample spacing between holesTM XP<0.001**5.935
Priming*TM XP*AM XP0.006**3.087
DFTM axiomMain effect or interactionP-valueF-value
Reduce part countTM XP*AM XP0.098*1.610
Low-labor-cost operationsPriming0.080*3.100
TM XP*AM XP0.078*1.689
Standard materials, components, and toolingTM XP0.014**3.208
Priming*TM XP*AM XP0.019**2.591
Uniform wall thicknessAM XP0.071*2.196
TM XP*AM XP0.099*1.604
Ample spacing between holesTM XP<0.001**5.935
Priming*TM XP*AM XP0.006**3.087

Note: *p < 0.1, **p < 0.05.

Table 6

Statistically significant DFAM axioms

DFAM axiomMain effect or interactionP-valueF-value
Complex shapes and geometriesAM XP0.035**2.635
Combining multiple parts into a single product or assemblyPriming*AM XP0.050*2.416
Avoiding large, flat regionsTM XP0.074*2.162
Orienting overhanging surfacesAM XP0.024**2.867
Considering the minimum feature sizeAM XP<0.001**5.045
Orienting curved surfacesAM XP0.035**2.632
Variations in material propertiesTM XP0.086*2.071
DFAM axiomMain effect or interactionP-valueF-value
Complex shapes and geometriesAM XP0.035**2.635
Combining multiple parts into a single product or assemblyPriming*AM XP0.050*2.416
Avoiding large, flat regionsTM XP0.074*2.162
Orienting overhanging surfacesAM XP0.024**2.867
Considering the minimum feature sizeAM XP<0.001**5.045
Orienting curved surfacesAM XP0.035**2.632
Variations in material propertiesTM XP0.086*2.071

Note: *p < 0.1, **p < 0.05.

The results in Tables 5 and 6 show that most of the 14 presented DFM axioms demonstrated statistically significant relationship with either TM experience, AM experience, the effect of priming, or some interaction of the three. A close examination of the significant findings shows that TM experience appears more frequently for the DFTM axioms than DFAM axioms, AM experience is roughly equally present across both sets of axioms, the effect of priming appears more frequently for the DFTM axioms, and the effect of priming is mostly associated with one of the two manufacturing experience variables.

To investigate where the specific differences in the self-reporting of the axioms were occurring within the manufacturing experience levels, a series of Tukey post-hoc tests were performed that compared the students’ self-reported scores for the significant axioms listed in Tables 5 and 6 against their manufacturing experience for both TM and AM. The post-hoc tests did not account for priming because while it did appear in the three-way ANOVA results shown in Tables 5 and 6, apart from one case, all the instances where priming was found to influence the self-reported scores were when priming was accompanied with manufacturing experience. This suggests that manufacturing experience has a more influential impact on the self-reported scores than priming. A 95% confidence interval was used to determine statistical significance (i.e., p < 0.05). The results of the Tukey post-hoc tests are presented in Tables 7 and 8 for DFTM axioms and DFAM axioms, respectively. The mean differences in self-reported scores are also reported, where a positive value indicates the lower manufacturing experience group on average assigned a higher score than the higher manufacturing experience group and a negative value indicates the higher manufacturing experience group on average assigned a higher score than the lower manufacturing experience group. Non-significant pairwise comparisons identified between manufacturing experience levels are indicated with “N/A.” Additionally, instances where statistical significance was observed with a 90% confidence interval (i.e., p < 0.1) were highlighted. While these cases may not be as significant to the same degree as the cases where significance was observed with a 95% confidence interval, based on the collected data it felt important to acknowledge these results as these relationships may become more or less significant (and therefore, more or less important) with additional data at the lowest and highest levels of manufacturing experience.

Table 7

Post-hoc results for DFTM axioms

DFTM axiomManufacturing experience typeLower manufacturing experience levelUpper manufacturing experience levelMean difference in scoresStandard deviation in scoresP-value
Reduce part countTMN/AN/AN/AN/AN/A
AMN/AN/AN/AN/AN/A
Low-labor-cost operationsTMN/AN/AN/AN/AN/A
AMN/AN/AN/AN/AN/A
Standard materials, components, and toolingTMN/AN/AN/AN/AN/A
AMN/AN/AN/AN/AN/A
Uniform wall thicknessTMN/AN/AN/AN/AN/A
AMN/AN/AN/AN/AN/A
Ample spacing between holesTM151.020.4120.098*
251.030.3560.035**
351.100.3560.019**
451.370.3670.002**
AMN/AN/AN/AN/AN/A
DFTM axiomManufacturing experience typeLower manufacturing experience levelUpper manufacturing experience levelMean difference in scoresStandard deviation in scoresP-value
Reduce part countTMN/AN/AN/AN/AN/A
AMN/AN/AN/AN/AN/A
Low-labor-cost operationsTMN/AN/AN/AN/AN/A
AMN/AN/AN/AN/AN/A
Standard materials, components, and toolingTMN/AN/AN/AN/AN/A
AMN/AN/AN/AN/AN/A
Uniform wall thicknessTMN/AN/AN/AN/AN/A
AMN/AN/AN/AN/AN/A
Ample spacing between holesTM151.020.4120.098*
251.030.3560.035**
351.100.3560.019**
451.370.3670.002**
AMN/AN/AN/AN/AN/A

Note: *p < 0.1, **p < 0.05.

Table 8

Post-hoc results for DFAM axioms

DFAM axiomManufacturing experience typeLower manufacturing experience levelUpper manufacturing experience levelMean difference in scoresStandard deviation in scoresP-value
Complex shapes and geometriesTM25−1.280.5080.090*
AM13−1.180.4440.066*
15−1.890.5850.013**
23−0.450.1800.097*
25−1.160.4220.050*
Combining multiple parts into a single product or assemblyTM15−1.520.5860.074*
AM15−2.040.5830.005**
25−1.430.4200.007**
Avoiding large, flat regionsTM15−1.480.5550.064*
25−1.620.4800.008**
35−1.500.4790.017**
45−1.330.4950.058*
AMN/AN/AN/AN/AN/A
Orienting overhanging surfacesTMN/AN/AN/AN/AN/A
AM15−2.180.5720.002**
25−1.580.4120.002**
35−1.270.4110.019**
45−1.150.4280.059*
Considering the minimum feature sizeTMN/AN/AN/AN/AN/A
AM13−0.840.3370.099*
14−1.200.3500.006**
15−1.930.445<0.001**
25−1.120.3200.005**
35−1.090.3200.007**
Orienting curved surfacesTMN/AN/AN/AN/AN/A
AMN/AN/AN/AN/AN/A
Variations in material propertiesTMN/AN/AN/AN/AN/A
AMN/AN/AN/AN/AN/A
DFAM axiomManufacturing experience typeLower manufacturing experience levelUpper manufacturing experience levelMean difference in scoresStandard deviation in scoresP-value
Complex shapes and geometriesTM25−1.280.5080.090*
AM13−1.180.4440.066*
15−1.890.5850.013**
23−0.450.1800.097*
25−1.160.4220.050*
Combining multiple parts into a single product or assemblyTM15−1.520.5860.074*
AM15−2.040.5830.005**
25−1.430.4200.007**
Avoiding large, flat regionsTM15−1.480.5550.064*
25−1.620.4800.008**
35−1.500.4790.017**
45−1.330.4950.058*
AMN/AN/AN/AN/AN/A
Orienting overhanging surfacesTMN/AN/AN/AN/AN/A
AM15−2.180.5720.002**
25−1.580.4120.002**
35−1.270.4110.019**
45−1.150.4280.059*
Considering the minimum feature sizeTMN/AN/AN/AN/AN/A
AM13−0.840.3370.099*
14−1.200.3500.006**
15−1.930.445<0.001**
25−1.120.3200.005**
35−1.090.3200.007**
Orienting curved surfacesTMN/AN/AN/AN/AN/A
AMN/AN/AN/AN/AN/A
Variations in material propertiesTMN/AN/AN/AN/AN/A
AMN/AN/AN/AN/AN/A

Note: *p < 0.1, **p < 0.05.

The results in Tables 7 and 8 show many instances where significant differences in the self-reporting of designs were observed across differing manufacturing experience levels. Most of the significant differences observed took place when the students were evaluating the designs when using DFAM axioms. In addition to these significant differences more frequently occurring when assessed based on AM experience, most of the mean difference score values were negative, indicating that on average the students with higher self-identified manufacturing experience more greatly applied the DFM axioms in their designs than the students with lower self-identified manufacturing experience.

6 Discussion

Based on the experimental results, there are several key findings that merit more in-depth discussion:

  • At low manufacturing experience levels, participants produced designs that were more suited for TM.

  • As AM experience increases, the number of designs suited for AM likewise increases.

  • Content priming can influence the traditional manufacturability of the designs.

  • Manufacturing experience can increase the self-reporting of DFM axioms in the design assessment process.

6.1 Students’ Designs Are More Suited for Traditional Manufacturing at Low Manufacturing Experience Levels.

H1.1 stated that designs from students at a low self-reported manufacturing experience level would tend toward TM. The results from Fig. 9 show that at low manufacturing experience levels (TM 1–2 and AM 1–2) the students in this study, regardless of the manufacturing process, created designs that were more suited for TM than AM. A lack of manufacturing experience across both processes forces the students to utilize any design considerations that may be subconsciously ingrained in their minds, which is most likely TM [29]. As an example, Fig. 10 shows three designs made by novice students (identifiers ENGE03, IAON06, and UELE03) who identified as having a low TM experience level (1, 2, and 1, respectively) and a low AM experience level (2, 2, and 1, respectively).

Fig. 10
Novice student designs
Fig. 10
Novice student designs
Close modal

From the raters, these designs received an average TM CAT score of 5.33, 4.67, and 3.67, respectively, and an average AM CAT score of 2.67, 3.33, and 2.33 respectively. The designs created by the novice students have simple characteristics, such as simple geometries, and minimize the number of parts in the design. For novices, these types of designs are anticipated because they do not yet possess the advanced knowledge to create complex designs. However, the common traits found in the designs of novice students are made up of the axioms that define DFTM, as will be discussed in Sec. 6.2; increasing manufacturing experience coincides with an increase in the number of designs suitable for AM.

6.2 Designs Become Suited for Additive Manufacturing Only as Additive Manufacturing Experience Increases.

H1.2 stated that increasing the manufacturing experience for either type of manufacturing (TM or AM) would increase the manufacturability score for that type of manufacturability (i.e., higher experience in DFAM would lead to designs that are more suitable for AM). The results in Fig. 9 show that as AM experience increased, the additive manufacturability of designs increased as well. As TM experience increased, however, there was not an increase in the traditional manufacturability of designs. Instead, the distribution of scores for additive manufacturability increased. This coincides with the findings in Table 4, which show that AM experience was found to have a higher correlation with both traditional manufacturability and additive manufacturability than the correlation associated with TM experience. This is a result of the challenges that come from learning new DFM axioms. These challenges can appear across all manufacturing experience levels [71].

In contrast to the designs created by the novice students, Fig. 11 shows three designs created by expert students (identifiers ENEK04, IUNG06, and ENIA07) who identified as having a high TM experience level (4, 4, and 4, respectively) and an above average AM experience level (3, 4, and 4, respectively). From the raters, these designs received an average TM CAT score of 2.67, 4.67, and 2.00, respectively, and an average AM CAT score of 4.67, 2.67, and 4.67, respectively.

Fig. 11
Expert student designs
Fig. 11
Expert student designs
Close modal

The designs created by the expert students demonstrate far more diversity than the designs created by the novice students, which may be attributed to these students having the experience necessary to produce a wider assortment of designs. By having the expertise necessary to implement these DFM axioms into their designs, these students can design for either manufacturing process by leveraging varying design characteristics such as simple geometries and ample spacing between holes for TM or they can choose to incorporate complex shapes and geometries for AM. Through examining which type of manufacturing experience is important to consider when looking at how these students are producing a wide variety of designs, the results in Table 4 indicate that the additive manufacturability of the designs that the students create is more closely related to their manufacturing experience than traditional manufacturability, regardless of the manufacturing experience type. This would indicate that in looking to address students’ designs for their manufacturability, it is better to evaluate the designs based on the students’ AM experience and adjust the feedback accordingly if the intent is to produce a design for TM or AM. Although the intent of these students in the ideation process was not captured, the findings suggest that possessing expert experience with the DFM axioms in both manufacturing processes enables the students to pick certain axioms to include in their designs because they have the capability to create a design that is best suited for one manufacturing process over another.

6.3 The Use of Priming Content Affects the Traditional Manufacturability of Participants’ Design Concepts.

H1.3 stated that students who receive the manufacturing priming content (for both TM and AM) will create designs that are better suited for manufacturing through either TM or AM (more specifically, the students would either use the DFTM priming content to create designs better suited for TM, or they would use the DFAM priming content to create designs better suited for AM). Tables 2 and 3 show that while the manufacturing priming content was found to significantly influence the designs’ traditional manufacturability, there were no significant instances observed regarding where these significant differences are occurring. This means that designs from students who received the manufacturing priming content were collectively different than those submitted by the students who did not receive the priming content.

In reviewing the average and standard deviation values in Table 1, the TM CAT scores from the students who received the manufacturing priming content were on average lower than the students who did not receive the priming content. This means that the priming content, which was intended to help improve the manufacturability of the designs across all levels of manufacturing experience, caused the designs to worsen in their manufacturability for TM. One possible reason for why the students who received the priming content may have decreased their TM CAT scores is a result of cognitive overload, which is when someone receives an overload of information in a short timeframe and cannot properly process the newly acquired information [72]. It may be possible that the students were overloaded with the DFM axioms and in turn created designs that used a mixture of DFTM and DFAM axioms, leading to poor designs for TM.

Another possible reason for the manufacturing priming content decreasing the designs’ TM CAT scores may be order bias. Also commonly referred to as recency bias, order bias is when decisions are made based on the overweighting of the most recent item presented [46]. It is plausible that because the DFAM axioms were presented to the students as part of the manufacturing priming immediately before the design challenge took place, the students may have tried to incorporate some of the axioms they were most recently exposed to into their designs, thereby worsening their traditional manufacturability. This would align with the work of Arnold et al. [73], who found that in conditions of heavy information load that order bias could influence decision making. These potential justifications, while certainly possible, cannot be confirmed in this work and would need to be investigated in a future study.

6.4 Manufacturing Experience Can Affect the Student’s Evaluation of Designs for Traditional Manufacturing or Additive Manufacturing.

H2.1 stated that students at a low self-reported manufacturing experience level would not apply DFM axioms in their designs as significantly as those at a high self-reported experience level. From the results, there were significant changes found across both sets of axioms. More specifically, while both TM and AM experience were found to have a significant effect on the self-reporting of DFTM and DFAM axioms, a connection to AM experience was more prevalent across both sets of axioms. This coincides with the DFAM axioms encompassing nearly all the significant DFM axioms found in the self-reporting process. These findings were solidified when accounting for the manufacturing priming content. H2.2 stated that students who receive the manufacturing priming content will more greatly apply DFM axioms in their designs. The results in Tables 5 and 6 show that the priming content rarely appeared as an influential factor in the self-assessment of designs relative to manufacturing experience. With nearly all instances of the manufacturing priming content appearing in Tables 5 and 6 being coupled with manufacturing experience, this further emphasizes the ability of the students to apply the DFM axioms being affected not by their exposure to priming content but by their manufacturing experience.

One justification for these findings comes from the possibility of the students having informal exposure to TM. By having informal exposure and training to TM, the students are already familiar with the DFTM axioms, resulting in fewer DFTM axioms showing significant differences of self-reporting compared to the DFAM axioms. While students may have started to become informally exposed to AM through accessible manufacturing facilities in the form of FabLabs [74], because AM is newer, most participants are likely to have not been exposed to the DFAM axioms. Presently, most students are not expected to have formal experience with the DFAM axioms because they are not incorporated in academic settings; there are growing calls for incorporating DFAM into curriculum, however, as stated by Prabhu et al. [75]. The students who do have formal self-reported experience produce significant differences for the self-reported DFAM axioms.

7 Conclusions and Future Work

In this study, an experiment was conducted to observe students’ design tendencies based on their previous manufacturing experience and having been presented with manufacturing priming content. Students completed a design challenge where the designs were assessed for manufacturability based on expert evaluation and self-assessment, where only some students received manufacturing priming content to bring the DFM axioms to the forefront of their minds. It was found that at low manufacturing experience levels, students’ designs are more suited for TM than AM. Additionally, informal TM experience meant that only significant changes were observed in the student’s self-reported use of DFAM axioms, along with an improvement in the designs’ additive manufacturability based on an increase in the AM experience level. Furthermore, it was found that the manufacturing priming content significantly influenced the traditional manufacturability of the designs. Lastly, the manufacturing priming content did not improve the students’ ability to recognize and identify the DFM axioms in their designs through self-assessment. These findings are important for the understanding of the students’ thought process as they progress through the early-stage design process. For students with low manufacturing experience levels, they are defaulting to using TM axioms based on their informal experience. For students with high manufacturing experience levels, they are making a sub-conscious decision to choose TM or AM since they have experience with both processes. Additionally, to cause a student to rethink their thought process in the concept generation stage there must be more work done than simply presenting the DFM axioms in a rapid environment.

We recognize that by overloading information onto the students with the design prompt without any prior knowledge about the requirements and asking them to generate a solution in a short timeframe that the experimental setup may have induced certain biases that had a stronger influence on the concept generation than the priming content. Given the time constraints, the experiment may have induced the complexity bias, which is when certain external factors increase the perceived complexity of a design problem [46,76,77] when the prompt may actually be straightforward to solve. As a result, the students may quickly rely on simple geometries for the perceived complex task, despite conceivable solutions being relatively simple. This immediately shifts the designs to be better suited for TM. If given extended time, the students may feel free to explore more geometrically complex solutions that would better align with AM. Additionally, because the students were asked to only submit hand-drawn sketches, there was no feasible way to produce a prototype of the design to evaluate its effectiveness. Without assurances on how the design would perform students may feel the need to create simple solutions that appear safer and to them can reliably perform the given task while meeting the requirements [46,78,79]. Lastly, the small sample size of participants at the lowest and highest levels of manufacturing experience, while disappointing for data collection, did not have an impact on the conclusions made regarding how the designs from students with low manufacturing experience compared to the students with high manufacturing experience. For future work, additional objective values of experience for TM and AM will be collected and analyzed to confirm the relationship between students’ self-reported experience and their objective experience. Future studies will also recruit more participants at the lowest and highest levels of manufacturing experience to make conclusions regarding how the designs vary across specific manufacturing experience levels (i.e., AM experience level 1 compared to AM experience level 2) and investigate the DFM axioms that students are using in their designs to evaluate what their natural tendencies are and the frequency with which these axioms are used.

Acknowledgment

This research was conducted through the support of the National Science Foundation under Grant No. 2042917. Any opinions, findings, and conclusions expressed in this paper are those of the authors and do not necessarily reflect the views of the NSF. We would also like to thank Dr. Randall Bock and Dr. Jason Moore for allowing us to conduct the experiment in their respective classes. Lastly, we would like to thank Jayant Mathur for helping with the statistics calculations.

Conflict of Interest

There are no conflicts of interest.

Data Availability Statement

The datasets generated and supporting the findings of this article are obtainable from the corresponding author upon reasonable request.

References

1.
Attaran
,
M.
,
2017
, “
The Rise of 3-D Printing: The Advantages of Additive Manufacturing Over Traditional Manufacturing
,”
Bus. Horiz.
,
60
(
5
), pp.
677
688
.
2.
Durakovic
,
B.
,
2018
, “
Design for Additive Manufacturing: Benefits, Trends and Challenges
,”
Period. Eng. Nat. Sci.
,
6
(
2
), pp.
179
191
.
3.
Prabhu
,
R.
,
Miller
,
S. R.
,
Simpson
,
T. W.
, and
Meisel
,
N. A.
,
2020
, “
Teaching Design Freedom: Understanding the Effects of Variations in Design for Additive Manufacturing Education on Students’ Creativity
,”
ASME J. Mech. Des.
,
142
(
9
), p.
094501
.
4.
Prabhu
,
R.
,
Miller
,
S. R.
,
Simpson
,
T. W.
, and
Meisel
,
N. A.
,
2020
, “
Exploring the Effects of Additive Manufacturing Education on Students’ Engineering Design Process and Its Outcomes
,”
ASME J. Mech. Des.
,
142
(
4
), p.
042001
.
5.
Yang
,
S.
,
Page
,
T.
, and
Zhao
,
Y. F.
,
2019
, “
Understanding the Role of Additive Manufacturing Knowledge in Stimulating Design Innovation for Novice Designers
,”
ASME J. Mech. Des.
,
141
(
2
), p.
021703
.
6.
Starkey
,
E.
,
Toh
,
C. A.
, and
Miller
,
S. R.
,
2016
, “
Abandoning Creativity: The Evolution of Creative Ideas in Engineering Design Course Projects
,”
Des. Stud.
,
47
, pp.
47
72
.
7.
Lamb
,
R.
,
Akmal
,
T.
, and
Petrie
,
K.
,
2015
, “
Development of a Cognition-Priming Model Describing Learning in a STEM Classroom
,”
J. Res. Sci. Teach.
,
52
(
3
), pp.
410
437
.
8.
Jung
,
Y.
,
Kim
,
Y.
, and
Murphy
,
J.
,
2017
, “
The Role of Task Repetition in Learning Word-Stress Patterns Through Auditory Priming Tasks
,”
Stud. Second Lang. Acquis.
,
39
(
2
), pp.
319
346
.
9.
Sanders
,
E. B.-N.
,
Brandt
,
E.
, and
Binder
,
T.
,
2010
, “
A Framework for Organizing the Tools and Techniques of Participatory Design
,”
PDC'10: Proceedings of the 11th Biennial Participatory Design Conference
,
Sydney, Australia
,
Nov. 29–Dec. 3
, pp.
195
198
.
10.
Liao
,
T.
, and
MacDonald
,
E. F.
,
2021
, “
Priming on Sustainable Design Idea Creation and Evaluation
,”
Sustainability
,
13
(
9
), p.
5227
.
11.
Viswanathan
,
V. K.
, and
Linsey
,
J. S.
,
2013
, “
Design Fixation and Its Mitigation: A Study on the Role of Expertise
,”
ASME J. Mech. Des.
,
135
(
5
), p.
051008
.
12.
Liu
,
Y.-C.
,
Chakrabarti
,
A.
, and
Bligh
,
T.
,
2003
, “
Towards an ‘Ideal’ Approach for Concept Generation
,”
Des. Stud.
,
24
(
4
), pp.
341
355
.
13.
Parnes
,
S. J.
, and
Meadow
,
A.
,
1959
, “
“Effects of” Brainstorming” Instructions on Creative Problem Solving by Trained and Untrained Subjects
,”
J. Educ. Psychol.
,
50
(
4
), pp.
171
176
.
14.
Li
,
Y.
,
Wang
,
J.
,
Li
,
X.
, and
Zhao
,
W.
,
2007
, “
Design Creativity in Product Innovation
,”
Int. J. Adv. Manuf. Technol.
,
33
(
3–4
), pp.
213
222
.
15.
Cross
,
N.
,
2004
, “
Expertise in Design: An Overview
,”
Des. Stud.
,
25
(
5
), pp.
427
441
.
16.
Ahmed
,
S.
,
Wallace
,
K. M.
, and
Blessing
,
L. T.
,
2003
, “
Understanding the Differences Between How Novice and Experienced Designers Approach Design Tasks
,”
Res. Eng. Des.
,
14
(
1
), pp.
1
11
.
17.
Daly
,
S. R.
,
Yilmaz
,
S.
,
Christian
,
J. L.
,
Seifert
,
C. M.
, and
Gonzalez
,
R.
,
2012
, “
Design Heuristics in Engineering Concept Generation
,”
J. Eng. Educ.
,
101
(
4
), pp.
601
629
.
18.
Ratcliff
,
R.
, and
McKoon
,
G.
, “
A Retrieval Theory of Priming in Memory
,”
Psychol. Rev.
,
95
(
3
), pp.
385
408
.
19.
Tulving
,
E.
, and
Schacter
,
D. L.
,
1990
, “
Priming and Human Memory Systems
,”
Science
,
247
(
4940
), pp.
301
306
.
20.
Schacter
,
D. L.
, and
Buckner
,
R. L.
,
1998
, “
Priming and the Brain
,”
Neuron
,
20
(
2
), pp.
185
195
.
21.
Bonnardel
,
N.
, and
Marmèche
,
E.
,
2004
, “
Evocation Processes by Novice and Expert Designers: Towards Stimulating Analogical Thinking
,”
Creat. Innov. Manag.
,
13
(
3
), pp.
176
186
.
22.
Yilmaz
,
S.
,
Christian
,
J. L.
,
Daly
,
S. R.
,
Seifert
,
C.
, and
Gonzalez
,
R.
,
2012
, “
How Do Design Heuristics Affects Outcomes?
,”
International Design Conference – Design 2012
,
Dubrovnik, Croatia
,
May 21–24
, pp.
1195
1204
.
23.
Atatreh
,
S.
,
Alyammahi
,
M.
,
Susantyoko
,
R. A.
,
Ismail
,
H.
, and
Mohammed
,
A.
,
2021
, “
Innovative Approaches to Enhance Awareness on Additive Manufacturing in Engineering Education Towards Competencies for Industry 4.0
,”
Volume 9: Engineering Education
,
Virtual, Online
,
American Society of Mechanical Engineers
, p.
V009T09A038
.
24.
Lauff
,
C. A.
,
Perez
,
K. B.
,
Camburn
,
B. A.
, and
Wood
,
K. L.
,
2019
, “
Design Principle Cards: Toolset to Support Innovations With Additive Manufacturing
,”
Volume 4: 24th Design for Manufacturing and the Life Cycle Conference; 13th International Conference on Micro- and Nanosystems
,
Anaheim, CA
,
American Society of Mechanical Engineers
, p.
V004T05A005
.
25.
Laverne
,
F.
,
Segonds
,
F.
,
Anwer
,
N.
, and
Le Coq
,
M.
,
2015
, “
Assembly Based Methods to Support Product Innovation in Design for Additive Manufacturing: An Exploratory Case Study
,”
ASME J. Mech. Des.
,
137
(
12
), p.
121701
.
26.
Perez
,
A.
,
Hölttä-Otto
,
K.
, and
Wood
,
K.
,
2015
, “
Crowdsourced Design Principles for Leveraging the Capabilities of Additive Manufacturing
.”
International Conference on Engineering Design, ICED15
,
Milan, Italy
,
July 27–30
, pp.
1
10
.
27.
Lough
,
K. G.
,
Stone
,
R.
, and
Tumer
,
I. Y.
,
2009
, “
The Risk in Early Design Method
,”
J. Eng. Des.
,
20
(
2
), pp.
155
173
.
28.
Yilmaz
,
S.
, and
Seifert
,
C. M.
,
2010
, “
Cognitive Heuristics in Design Ideation
,”
DS 60: Proceedings of DESIGN 2010, the 11th International Design Conference
,
Dubrovnik, Croatia
,
May 17–20
, pp.
1007
1016
.
29.
Blösch-Paidosh
,
A.
, and
Shea
,
K.
,
2019
, “
Design Heuristics for Additive Manufacturing Validated Through a User Study
,”
ASME J. Mech. Des.
,
141
(
4
), p.
041101
.
31.
Blösch-Paidosh
,
A.
,
Ahmed-Kristensen
,
S.
, and
Shea
,
K.
,
2019
, “
Evaluating the Potential of Design for Additive Manufacturing Heuristic Cards to Stimulate Novel Product Redesigns
,”
Volume 2A: 45th Design Automation Conference
,
Anaheim, CA
,
American Society of Mechanical Engineers
, p.
V02AT03A036
.
32.
Watschke
,
H.
,
Bavendiek
,
A.-K.
,
Giannakos
,
A.
, and
Vietor
,
T.
,
2017
, “
A Methodical Approach to Support Ideation for Additive Manufacturing in Design Education
,”
DS 87-5 Proceedings of the 21st International Conference on Engineering Design (ICED 17) Vol 5: Design for X, Design to X
,
Vancouver, Canada
,
Aug. 21–25
, pp.
041
050
.
33.
Floriane
,
L.
,
Frédéric
,
S.
,
Gianluca
,
D.
, and
Marc
,
L. C.
,
2017
, “
Enriching Design With X Through Tailored Additive Manufacturing Knowledge: A Methodological Proposal
,”
Int. J. Interact. Des. Manuf.
,
11
(
2
), pp.
279
288
.
34.
Prabhu
,
R.
,
Leguarda
,
R. L.
,
Miller
,
S. R.
,
Simpson
,
T. W.
, and
Meisel
,
N. A.
,
2021
, “
Favoring Complexity: A Mixed Methods Exploration of Factors That Influence Concept Selection When Designing for Additive Manufacturing
,”
ASME J. Mech. Des.
,
143
(
10
), p.
102001
.
35.
Prabhu
,
R.
,
Miller
,
S. R.
,
Simpson
,
T. W.
, and
Meisel
,
N. A.
,
2020
, “
But Will It Build? Assessing Student Engineering Designers’ Use of Design for Additive Manufacturing Considerations in Design Outcomes
,”
ASME J. Mech. Des.
,
142
(
9
), p.
092001
.
36.
Sinha
,
S.
,
Chen
,
H.-E.
,
Meisel
,
N. A.
, and
Miller
,
S. R.
,
2017
, “
Does Designing for Additive Manufacturing Help Us Be More Creative? An Exploration in Engineering Design Education
,”
Volume 3: 19th International Conference on Advanced Vehicle Technologies; 14th International Conference on Design Education; 10th Frontiers in Biomedical Devices
,
Cleveland, OH
,
American Society of Mechanical Engineers
, p.
V003T04A014
.
37.
Seepersad
,
C. C.
,
2014
, “
Challenges and Opportunities in Design for Additive Manufacturing
,”
3D Print. Addit. Manuf.
,
1
(
1
), pp.
10
13
.
38.
Novak
,
E.
,
Brannon
,
M.
,
Librea-Carden
,
M. R.
, and
Haas
,
A. L.
,
2021
, “
A Systematic Review of Empirical Research on Learning With 3D Printing Technology
,”
J. Comput. Assist. Learn.
,
37
(
5
), pp.
1455
1478
.
39.
Swamidass
,
P. M.
, and
Winch
,
G. W.
,
2002
, “
Exploratory Study of the Adoption of Manufacturing Technology Innovations in the USA and the UK
,”
Int. J. Prod. Res.
,
40
(
12
), pp.
2677
2703
.
40.
Chryssolouris
,
G.
,
Mavrikios
,
D.
, and
Rentzos
,
L.
,
2016
, “
The Teaching Factory: A Manufacturing Education Paradigm
,”
Proc. CIRP
,
57
, pp.
44
48
.
41.
Salter
,
A.
, and
Gann
,
D.
,
2003
, “
Sources of Ideas for Innovation in Engineering Design
,”
Res. Policy
,
32
(
8
), pp.
1309
1324
.
42.
Janiszewski
,
C.
, and
Wyer Jr
,
R. S.
,
2014
, “
Content and Process Priming: A Review
,”
J. Consum. Psychol.
,
24
(
1
), pp.
96
118
.
43.
Valjak
,
F.
, and
Lindwall
,
A.
,
2021
, “
Review of Design Heuristics and Design Principles in Design for Additive Manufacturing
,”
Proc. Des. Soc.
,
1
, pp.
2571
2580
.
44.
Lindwall
,
A.
, and
Törlind
,
P.
,
2018
, “
Evaluating Design Heuristics for Additive Manufacturing as an Explorative Workshop Method
,”
International Design Conference – Design 2018
, pp.
1221
1232
.
45.
Duverger
,
P.
,
2012
, “
Using YouTube Videos as a Primer to Affect Academic Content Retention
,”
Metrop. Univ.
,
23
(
2
), pp.
51
66
.
46.
Arnott
,
D.
,
2006
, “
Cognitive Biases and Decision Support Systems Development: A Design Science Approach
,”
Inf. Syst. J.
,
16
(
1
), pp.
55
78
.
47.
Yates
,
J. F.
, and
Curley
,
S. P.
,
1986
, “
Contingency Judgment: Primacy Effects and Attention Decrement
,”
Acta Psychol.
,
62
(
3
), pp.
293
302
.
48.
Chapman
,
G. B.
,
Bergus
,
G. R.
, and
Elstein
,
A. S.
,
1996
, “
Order of Information Affects Clinical Judgment
,”
J. Behav. Decis. Mak.
,
9
(
3
), pp.
201
211
.
49.
Joshi
,
A.
,
Kale
,
S.
,
Chandel
,
S.
, and
Pal
,
D. K.
,
2015
, “
Likert Scale: Explored and Explained
,”
Br. J. Appl. Sci. Technol.
,
7
(
4
), pp.
396
403
.
50.
“Files—Made By Design Lab,” https://sites.psu.edu/madebydesign/files/, Accessed February 14, 2023.
51.
Gordon
,
M. J.
,
1991
, “
A Review of the Validity and Accuracy of Self-Assessments in Health Professions Training
,”
Acad. Med.
,
66
(
12
), pp.
762
769
.
52.
Prabhu
,
R.
,
Simpson
,
T. W.
,
Miller
,
S. R.
, and
Meisel
,
N. A.
,
2022
, “
Development and Validity Evidence Investigation of a Design for Additive Manufacturing Self-Efficacy Scale
,”
Res. Eng. Des.
,
33
(
4
), pp.
437
453
.
53.
Bralla
,
J. G.
,
1999
,
Design for Manufacturability Handbook
,
McGraw-Hill Education
,
New York
.
54.
Prabhu
,
R.
,
Bracken
,
J.
,
Armstrong
,
C. B.
,
Jablokow
,
K.
,
Simpson
,
T. W.
, and
Meisel
,
N. A.
,
2020
, “
Additive Creativity: Investigating the Use of Design for Additive Manufacturing to Encourage Creativity in the Engineering Design Industry
,”
Int. J. Des. Creat. Innov.
,
8
(
4
), pp.
198
222
.
55.
Williams
,
C. B.
, and
Seepersad
,
C. C.
,
2012
, “
Design for Additive Manufacturing Curriculum: A Problem-and Project-Based Approach
,”
University of Texas at Austin
,
Austin, TX
.
56.
Stern
,
A.
,
Rosenthal
,
Y.
,
Dresler
,
N.
, and
Ashkenazi
,
D.
,
2019
, “
Additive Manufacturing: An Education Strategy for Engineering Students
,”
Addit. Manuf.
,
27
, pp.
503
514
.
57.
Sullivan
,
G. M.
, and
Artino
,
A. R.
, Jr.
,
2013
, “
Analyzing and Interpreting Data From Likert-Type Scales
,”
J. Grad. Med. Educ.
,
5
(
4
), pp.
541
542
.
58.
Amabile
,
T. M.
,
1982
, “
Social Psychology of Creativity: A Consensual Assessment Technique
,”
J. Pers. Soc. Psychol.
,
43
(
5
), pp.
997
1013
.
59.
Baer
,
J.
, and
McKool
,
S.
,
2009
, “Assessing Creativity Using the Consensual Assessment Technique,”
Handbook of Research on Assessment Technologies, Methods, and Applications in Higher Education
,
C. S.
Schreiner
, ed., pp.
65
77
.
60.
Prabhu
,
R.
,
Masia
,
J. S.
,
Berthel
,
J. T.
,
Meisel
,
N. A.
, and
Simpson
,
T. W.
,
2021
, “
Maximizing Design Potential: Investigating the Effects of Utilizing Opportunistic and Restrictive Design for Additive Manufacturing in Rapid Response Solutions
,”
Rapid Prototyp. J.
,
27
(
6
), pp.
1161
1171
.
61.
Lorenz
,
K.
,
Jones
,
J.
,
Wimpenny
,
D.
, and
Jackson
,
M.
,
2015
, “
A Review of Hybrid Manufacturing
,”
University of Texas at Austin
,
Austin, TX
.
62.
Bartko
,
J. J.
,
1966
, “
The Intraclass Correlation Coefficient as a Measure of Reliability
,”
Psychol. Rep.
,
19
(
1
), pp.
3
11
.
63.
Weir
,
J. P.
,
2005
, “
Quantifying Test-Retest Reliability Using the Intraclass Correlation Coefficient and the SEM
,”
J. Strength Cond. Res.
,
19
(
1
), pp.
231
240
.
64.
Fleiss
,
J. L.
, and
Cohen
,
J.
,
1973
, “
The Equivalence of Weighted Kappa and the Intraclass Correlation Coefficient as Measures of Reliability
,”
Educ. Psychol. Meas.
,
33
(
3
), pp.
613
619
.
65.
Lee
,
J.
,
Koh
,
D.
, and
Ong
,
C.
,
1989
, “
Statistical Evaluation of Agreement Between Two Methods for Measuring a Quantitative Variable
,”
Comput. Biol. Med.
,
19
(
1
), pp.
61
70
.
66.
Genco
,
N. E.
,
Holtta-Otto
,
K.
, and
Seepersad
,
C. C.
,
2011
, “
Factors That Influence the Creativity of Engineering Students
,”
ASEE Annual Conference
,
Vancouver, Canada
,
June 26–29
, pp.
1
15
.
67.
Taylor
,
R.
,
1990
, “
Interpretation of the Correlation Coefficient: A Basic Review
,”
J. Diagn. Med. Sonogr.
,
6
(
1
), pp.
35
39
.
68.
Hanusz
,
Z.
,
Tarasinska
,
J.
, and
Zielinski
,
W.
,
2016
, “
Shapiro–Wilk Test With Known Mean
,”
REVSTAT-Stat. J.
,
14
(
1
), pp.
89
100
.
69.
Mircioiu
,
C.
, and
Atkinson
,
J.
,
2017
, “
A Comparison of Parametric and Non-Parametric Methods Applied to a Likert Scale
,”
Pharmacy
,
5
(
2
), p.
26
.
70.
Murray
,
J.
,
2013
, “
Likert Data: What to Use, Parametric or Non-Parametric?
,”
Int. J. Bus. Soc. Sci.
,
4
(
11
), pp.
258
264
.
71.
Dinar
,
M.
, and
Rosen
,
D. W.
,
2017
, “
A Design for Additive Manufacturing Ontology
,”
ASME J. Comput. Inf. Sci. Eng.
,
17
(
2
), p.
021013
.
72.
Kirsh
,
D.
,
2000
, “
A Few Thoughts on Cognitive Overload
,”
Intellectica
,
1
(
30
), pp.
19
51
.
73.
Arnold
,
V.
,
Collier
,
P. A.
,
Leech
,
S. A.
, and
Sutton
,
S. G.
,
2000
, “
The Effect of Experience and Complexity on Order and Recency Bias in Decision Making by Professional Accountants
,”
Account. Finance
,
40
(
2
), pp.
109
134
.
74.
Blikstein
,
P.
, and
Krannich
,
D.
,
2013
, “
The Makers’ Movement and FabLabs in Education: Experiences, Technologies, and Research
,”
Proceedings of the 12th International Conference on Interaction Design and Children
,
New York, NY
,
June 24–27
,
ACM
, pp.
613
616
.
75.
Prabhu
,
R.
,
Miller
,
S. R.
,
Simpson
,
T. W.
, and
Meisel
,
N. A.
,
2018
, “
The Earlier the Better? Investigating the Importance of Timing on Effectiveness of Design for Additive Manufacturing Education
,”
American Society of Mechanical Engineers
, p.
V02AT03A041
.
76.
Maule
,
A. J.
, and
Edland
,
A. C.
,
2002
, “The Effects of Time Pressure on Human Judgement and Decision Making,”
Decision Making
,
G.
Wright
, ed.,
Routledge
, pp.
203
218
.
77.
Ordóñez
,
L.
, and
Benson
,
L.
,
1997
, “
Decisions Under Time Pressure: How Time Constraint Affects Risky Decision Making
,”
Organ. Behav. Hum. Decis. Process.
,
71
(
2
), pp.
121
140
.
78.
Einhorn
,
H. J.
,
1980
, “
Learning From Experience and Suboptimal Rules in Decision Making
,”
Cognitive Processes in Choice and Decision Behavior
.
79.
Christensen-Szalanski
,
J. J.
, and
Bushyhead
,
J. B.
,
1981
, “
Physicians’ Use of Probabilistic Information in a Real Clinical Setting
,”
J. Exp. Psychol. Hum. Percept. Perform.
,
7
(
4
), pp.
928
935
.