Embedding Experiential Design Knowledge in Interactive Knowledge Graphs

Design can be considered a learning process, in which knowledge is collected, synthesized, and organized to achieve an outcome [1–3]. Representations of knowledge, its organization, and transformation underpin foundational models of the engineering design process, such as the function-behavior-structure (FBS) model and C-K design theory [4,5], but the importance of knowledge in design is not simply theoretical. By Robinson’s accounting, engineers spend more than 55% of their work hours acquiring or sharing knowledge [6], making knowledge organization, and structuring a critical element of engineering design practice. This is perhaps most evidenced from the importance of knowledge structuring, organization, and sharing in organizations, where it is considered a critical strategic focus and a basis of competitive advantage [7,8].

Numerous studies have examined how knowledge structuring affects the outcomes of design activities such as idea generation [9] and user feedback [10]. However, to effectively structure their knowledge, engineers and designers must navigate a large space of complex information [11,12], reconcile it with collaborators [13], and manage it toward a singular design outcome [14]. Despite many advances in virtual collaboration, remote work exacerbates these challenges [15]. This work seeks to explore virtual collaboration given the rise of remote work due to the effects of the ongoing COVID-19 pandemic [16].

Efforts to codify and structure engineering knowledge through knowledge graphs (KGs) have been very successful. KGs are networks of data containing nodes of information and edges, which store the relationships connecting various nodes. TechNet² is a leading example of a successful engineering KG. However, graphs like TechNet’s require well-structured semantic data [17–19], which may be considered explicit knowledge, or knowledge that is readily expressible and transferable [20]. Much of design knowledge, however, results from design activities, such as prototyping or teardowns and is not easily structured in an explicit manner. Such knowledge may be considered implicit, meaning that it exists internally to a designer, or tacit, meaning that it is not readily expressed externally [20]. Much of design knowledge, then, can be considered a result of what the Kolb Learning Model describes as “experiential learning” [21]. Combining the experiential nature of design activity data with the organizational advantages of a KG could address key challenges in knowledge structuring and sourcing at both the designer and organization levels.

Within Kolb’s model, the intersection of active experimentation and concrete experience describes a wide range of design activities [1], including reverse engineering, a rich source of knowledge for designers [22]. Reverse engineering allows designers to ascertain a product’s structure, function, and behavior afforded [22–24]. This empowers design teams to develop models and analyses based on the form and function of products, ultimately enabling them to design or redesign products [25]. Central to reverse engineering is the product teardown, also known as product dissection, in which designers disassemble and analyze an existing product and its constituent parts [22]. Numerous studies have described how teardowns help designers and design teams acquire knowledge [26–28], but little is captured in academic literature about how professional designers organize information emerging during and after teardowns. Understanding designers’ knowledge organization and structuring behavior is essential as products become increasingly complex, making their function, structure, and behavior increasingly accessible only to designers or groups of designers with specialized domain expertise [29,30].

In this work, we explore how experienced designers structure and organize knowledge and how KGs can be adapted to capture and use implicit or tacit knowledge for future design activities. We seek to explore two research questions:

• R1. How do designers organize experiential knowledge in teardowns?

• R2. How can designers’ knowledge organization behavior inform the construction of knowledge graphs?

To address these questions, we examine how designers and engineers organize design knowledge from a product teardown by conducting a participatory study with 23 design professionals. Then, we develop a KG based on this experiential data drawn from a real design activity. We demonstrate the KG with two extended examples. The main contributions of this work are (1) presenting insights into professional designers’ knowledge organization practices and (2) presenting a novel KG grounded in experiential design data. These findings represent a preliminary step toward design knowledge bases that more accurately reflect designer behavior and enable knowledge organization across teams. Ultimately, this work seeks to support data-driven design tools for organizing and availing experiential design knowledge emerging from complex design activities.

Knowledge organization and structuring can be considered core activities of the engineering design process [1,22]. A designer’s ability to incorporate and structure new knowledge has been demonstrated to uniquely shape innovative design outcomes [31]. Organizing design knowledge is essential not just for design results, but fundamentally shapes what Harfield calls the “problematization of design”: how a designer reconciles existing and new knowledge with a given problem frame, ultimately creating an addressable design problem [32]. How knowledge is organized during the design process shapes not just the process itself and its immediate outcomes, but its transition to a finished product, e.g., through product architecture [33].

At a macro-level, in the engineering design field, knowledge has been represented in large semantic networks, or knowledge bases, which represent knowledge via entities and relations interconnected in a graph structure [34,35]. Knowledge bases have served as key enablers for data-driven design methods [36]. Efforts have been made to create more engineering design-specific knowledge bases, for example, by mining technical publications, patent databases, or through manual labeling of design knowledge [17,37,38]. With their continued development and expansion, these design knowledge bases promise to support further data-driven product design methods, such as assessing similarity between design components, providing vector representations or embeddings of design concepts, or supporting functional modeling [36,39,40].

At a micro-level, designers have been shown to structure information in a variety of ways, from narratives [41,42] to sketches [43]. Of paramount importance across all of these approaches is the linking and grouping of knowledge in design [31], behavior which is foundational to both rules-based design [44] and less structured, e.g., “innovative” design [45], approaches. In Damen and Toh’s recent study of experienced design professionals’ knowledge structuring activity during idea generation, three modes of organization were observed: clusters, relations, and nests, each describing a unique way of linking different types of information [46]. The researchers found that while participant experience and discipline did not determine their mode of organization, the mode of organization was related to elements of design ideation results. Thus, how designers develop relationships between knowledge appears to shape the outcome of design activity. Earlier work by Le Masson et al. suggested that engineering design appeared to prioritize a much more systematic and rigorous structuring of linked knowledge than architectural design, highlighting the importance of linking and grouping in engineering design [47]. Beyond knowledge immediately relevant to the design task, analogies and similar knowledge have been shown to shape how designers reach a design outcome, even when such similarity is distant [48,49]. C-K design theory distinguishes knowledge (K) from concepts (C) as core elements to explain design reasoning. Briefly, as Hatchuel and Weil write, “knowledge” represents propositions that already exist that can be determined true or false. In contrast, “concepts” are what Hatchuel and Weil describe as “undecidable”—meaning, their truth or falsehood can’t be determined. Thus, by C-K design theory, design is a process of transforming concepts (e.g., the possibility of a product with particular properties) into other concepts or ultimately knowledge (e.g., a product that possesses these properties) [5,50]. In this process, knowledge spaces enable designers to develop new design concept, what the theory refers to as a K-C transformation or disjunction [50]. In contrast, linking between elements in a design knowledge space is also a fundamental operation in design reasoning, known as a K-K operator or expansion: forming links to expand the knowledge space through optimization, deduction, or other activities [5].

A foundational framework for understanding knowledge during the design process is the FBS model, which combines an ontology for understanding design knowledge [51,52] with consideration of a designer’s cognition and experience. The FBS framework has been applied to manage knowledge in a diverse range of activities across the engineering design process, from information extraction from patent databases [53] to defining product requirements [54]. Function describes what a designed object is for, or its purpose; for example, in Qian and Gero’s foundational article establishing the FBS framework, the authors describe an umbrella’s function to be blocking raindrops [4]. Behavior describes what a designed object does or its attributes; for example, Qian and Gero describe the behavior of fluid flow through a faucet to be characterized by a variable, flowrate [4]. Structure describes the components of an object and their relationship; for example, Qian and Gero describe the structure of a chair as consisting of a seat, four legs, and a back [4,52].

With the established importance of knowledge structuring and organization on design, numerous studies have explored how knowledge structuring and reconciliation shapes core design activities including idea generation [31,46], developing insights [55], communicating outcomes [10], and production activities in makerspaces [56]. Despite this breadth of research, few studies of knowledge structuring and organization during product teardowns have been reported. Many studies have explored knowledge and learning outcomes of reverse engineering tasks [28], but little is known about the process of organizing and structuring knowledge that enables such outcomes.

This study extends from previous research on design knowledge organization in two ways. First, we study knowledge organization during a virtual teardown activity, allowing us to understand how professional designers structure knowledge and information as they engage with teardowns. Second, we examine knowledge organization through two complementary lenses. We use the FBS framework as an ontology to describe types of knowledge content represented by information from a virtual teardown, and also as a prompt for designers to guide their grouping activity. We next use linking and grouping behaviors to identify knowledge relations. By examining knowledge organization during teardowns through these lenses, we can uniquely understand knowledge organization activities and abstract broad patterns from them.

KGs can be defined as networks that acquire and integrate information into an ontology which is then used to derive new knowledge [57]. KGs and semantic networks have long served as important references for large sets of general information (e.g., Google, etc.), but more recently have begun playing a large role in engineering, helping accelerate innovation and design. Databases like TechNet and ConceptNet³ are designed to hold vast arrays of technical data and meet growing knowledge retrieval and sharing needs [17,18]. Such semantic networks use natural language processing (NLP) techniques to collect data from large databases like the US Patent network (TechNet) and consolidate them into a single tool. Modeling this information in a multidomain KG that is easily navigated by algorithms enables users to access large amounts of interconnected technical data and drives novel, innovative solutions [58].

Among this range of existing KGs, semantic versus experiential graph types are particularly interesting for design, given the high degree of design knowledge that results from activities and interaction. The databases mentioned above are built by extracting a large data corpus and mapping it onto a proposed ontology. While mining large databases like the patent database produces thorough semantic networks, they have limited application and can be difficult to navigate [59] at a purely semantic level. Bhatia et al. explore the importance of adding descriptive support to KGs in order to add context and support user interaction [60]. By building our KG from a detailed, interactive experience, the data are supplemented by descriptions and low-level detail that helps situate knowledge, which has been shown to aid users’ understanding during information retrieval [61].

KGs have been used to support data-driven engineering design [36,59] as well as collaboration amongst large groups like companies [62]. This particular type of KG’s relevance in design has been shown to offer strong insights into product-level design [58], whereas our study explores the utilization of KGs in systems design, where various electromechanical elements are working in conjunction with one another. KGs have also begun appearing as efforts to effectively transfer design knowledge [63], a task that has been shown to be highly dependent on existing structures or practices [12]. Furthermore, Song and Fu showed the importance of visual interactions for seeking inspiration and supporting exploration, validating the use of KGs for user-driven exploratory search [64].

In this work, we extend upon prior research studying knowledge organization in virtual product teardowns through representation and exploration of our dataset within a KG. Unlike previous KGs built for engineering design applications, our graph captures experiential details collected during the teardown activity, offering in-depth insights into what relationships people create on the same set of knowledge. Additionally, we capture the roles of the participants who contributed to the data in the KG, imparting a level of participant diversity and nuance to the graph [65]. We use the unstructured teardown data to inform the structure of our graph, which can then query domain-specific ontologies constructed through a real design activity. We are able to run user-driven queries on this graph that are systems and large-scale design-behavior specific.

Reverse engineering is a cornerstone activity of professional design work, practiced by engineers in sectors ranging from software to machine design [66–68] as a way to ascertain an existing product’s structure, function, and value created, ultimately affording the design of new products [22,24]. The product teardown, or product dissection, is a core component of the reverse engineering of physical products, and involves, as Dalrymple et al. describe it, “a systematic deconstruction of an artefact, and the subsequent analysis … of its components for the purpose of understanding [its] physical, technological, and developmental principles” [28].

Studies of teardowns in companies have focused on descriptive accounts of how teardowns integrate with a broader product development process. Lauff et al., in their study of methods employed in firms engaging in early-stage product development, observed that companies developing consumer electronics and medical device products leveraged product teardowns during concept generation, while a company developing footwear products did not [69]. Morgan and Liker describe the role of teardowns in the Toyota Production System’s approach to product development [70]. However, despite the teardown’s centrality in engineering practice, few systematic studies have explored how companies and professional engineers and designers engage with teardowns, and specifically, how they generate and organize knowledge from teardowns.

The product teardown is widely used in engineering education, where it is prized for its experiential learning and preparation of students for industry [71,72]. Accordingly, its usage in classroom studies has been extensively studied. Teardowns in classroom contexts have been shown to help students understand the relationships between components of a product, and relate products to each other within product families [73]. Recent work has shown that virtual teardowns yield similar knowledge as in-person teardowns [74] and that augmenting virtual teardowns with rich interactions holds promise for improved outcomes [75]. Explorations of teardowns in engineering education provide a robust foundation for studies of professional designers, who are known to differ from students in their engagement with design process [76] while leveraging cross-disciplinary collaborations [13] and significant experience in manufacturing and product development [77] in their work.

In this work, we extend upon prior research on reverse engineering and product teardowns in two ways.

First, we specifically examine knowledge organization behavior during teardowns, building on previous studies of knowledge acquired from teardowns. This allows us to examine relationships between knowledge that emerge from the active experimentation inherent in a virtual teardown activity, helping us understand teardowns from a knowledge perspective. Second, we study professional designers with an average experience of more than three years, collecting unique insights on their relationship to product teardowns, and how professional designers in industry environments engage with knowledge generated during teardowns.

An overview of this work’s research methodology can be seen in Fig. 1.

Our teardown examined the Bose Tenor Frames, smart sunglasses with integrated speakers and a user interaction component. This product was chosen for its complexity and mix of mechanical and electrical components. Three volunteers from the Teardown Library⁴ performed and documented the teardown.⁵

One mechanical engineer with a background in smart glasses design initiated the teardown, documenting product highlights, the disassembly process, and observations. This individual then organized the observations based on the bill of materials hierarchy. Then, two other volunteers with electrical and mechanical engineering and product management backgrounds added more observations relevant to their domains. The term knowledge in our study refers to individual notes and images, as shown in Fig. 2. Following this knowledge extraction stage, the research team took the knowledge generated by the teardown volunteers and curated them to improve clarity, replace short-hand technical terminology, and reduce the number to an appropriate amount given the time restrictions of the study. This process of knowledge extraction and curation resulted in 24 images and 77 text notes to be used for the study (examples are shown in Fig. 2). The images and text described above were then placed in Mural,⁶ a digital collaboration tool used for the study.

A separate set of study participants were asked to organize the teardown knowledge in a guided think-aloud session. Participants were provided with the same teardown knowledge (as described in Sec. 3.1) and asked to perform four tasks: grouping the notes, linking the notes, linking the groups, and rating the links. Afterward, the participants also answered a survey to collect qualitative data about the tasks they performed. Having participants work with data from a teardown they did not perform more accurately represents how organizations engage with teardown knowledge: insights from teardowns are used by many in the organization who did not participate in the teardown activity.

Twenty-three professional designers were recruited for this study, through mailing lists and study postings. By recruiting professionals with various backgrounds, the study captures design priorities driven by multidisciplinary factors, such as manufacturing constraints and marketing demands, which professionals are exposed to over their careers. A summary of the role and average years of experience of the participants is shown in Table 1. The variability in domain expertise and role will allow us to learn how different groups organize knowledge differently. While teardowns are a subjective, participant-specific activity, Kearney et al.’s work shows that in the case of engineers performing teardowns on a given product, less variation appears in knowledge extracted within treatment groups than across treatment groups given different teardown modalities [78]. Thus, it appears that differences in participants’ subjective knowledge extraction are less than differences in the format of the teardown, suggesting that knowledge extracted from a teardown is relatively consistent within a given teardown modality.

The experiment was conducted on Mural, chosen to enable the researchers to perform the study remotely but synchronously, and to streamline data collection. Before the study, participants familiarized themselves with the online tool and practiced the required functionality.

After a brief introduction of the tasks, and the Mural learning session, the participants had 5 min to read through the raw teardown notes. The participants were given 60 min for the grouping, note and image linking, group linking, and rating tasks, while thinking out loud to explain their decisions. An example of these actions is shown in Fig. 2.

First, participants were asked to group similar notes and images and to assign a name to each group. Subgroups were allowed. The participants were instructed to group the notes and images based on three prompt variations: grouping by function, behavior, and structure. The assignment is randomized. Of the participants, seven were instructed to group the notes and images by function, eight by behavior, and eight by structure. Second, participants were prompted to link related notes and images to each other by drawing an arrow between the notes that had been previously grouped. They were also asked to label the links and to think out loud about the reasoning behind this connection. Third, participants were asked to find and draw links between groups they had created, and to label these connections with a descriptive name. Fourth, participants were asked to rate the links they had drawn from one to five (slightly related to very related). Following the completion of these tasks, participants were asked to complete a survey to report demographic information and reflect on the usefulness of the tasks for learning and knowledge transfer.

In the survey, we ask the designer to rate the usefulness of each activity in learning about smart glasses from one—not useful, to five—very useful, and explain why they give such rating with examples. Then we ask them to rate the usefulness of each activity in sharing knowledge about smart glasses with their team from one to five and explain why.

For each prompt variation (function, behavior, structure), here is the detailed breakdown of prompt assignment by roles. For behavior, there were four Mechanical Engineers, one Manufacturing Engineer, one Industrial Designer, one Systems Engineer, and one CEO/Manager. For function, there were three Mechanical Engineers, three Manufacturing Engineers, one CEO/Manager, and one Electrical Engineer. For structure, there were three Mechanical Engineers, two Manufacturing Engineers, one Industrial Designer, and one Systems Engineer.

Mural diagrams and survey results were registered into a spreadsheet for further analysis to prepare for coding. Notes and images were coded using the FBS ontology. Three researchers independently categorized each note, using rules to guide coding (Table 2). These rules are developed around the definitions and examples cited in Sec. 2.1. Notably, we distinguish structure attributes from behavior variables by arguing that dimensions, weight, etc. of the product, subassembly, or other components are what Qian and Gero consider behavior variables derived directly or indirectly from structural factors [4]. After triple-coding, initial inter-rater reliability (IRR), calculated as the total number of agreements divided by the total number of ratings [79], was determined to be 58%. As this figure was less than an IRR threshold of 80%, the coders discussed their disagreements and resolved differences to reach a 100% IRR. These resolved and agreed-upon codes are the final codes presented and used for further analysis.

After the data were encoded, groups with similar descriptions were consolidated before analyzing the data.

To do so, keywords were extracted from the group names using NLP with the TechNet API.⁷ TechNet is a semantic network of technical terms that was used to identify higher-level semantic names from the participant-created group names [17]. Three researchers independently associated participant-created groups with consolidated groups using the TechNet-generated group names, and added additional names. In total, 159 individual groups were consolidated into 28 new groups.

Additional coding was performed to support KG construction: for the input into the knowledge graph, the descriptions of links were consolidated. The categories of links and frequencies assigned are summarized in Table 3. Two categories of note are Collaboration and Tradeoff, which describe relationships where two nodes have elements which either rely on one another or must be sacrificed for one another, respectively. Individual notes were classified using an additional category for labels that described the knowledge itself, a set of descriptions that was not present for group links. To categorize the links, two researchers independently classified all links according to the categories, before resolving discrepancies. Examples of these link classifications can be found in Table 4.

In order to create a KG from the teardown data, spreadsheets containing study data (including raw notes, images, and links) were formatted and imported as a .csv file into Neo4j,⁸ an open-source graph database management system. Next, mappings were made to properties within the spreadsheets, so that they could be effectively attached to nodes and edges in the graph. Using Neo4j’s native Cypher query language, the data were converted from .csv files into nodes and edges within the Neo4j KG. Additional details about the teardown from the spreadsheets were attached as properties of nodes and edges (Sec. 4.2.2). Finally, the graph in Neo4j was exported as a .dump file, to be used in the Neo4j Bloom interface⁹ for visualizing various interaction modes. This tool provides near-natural language search on the KG and intuitive navigation not explicitly requiring Cypher code, easing graph navigation and representation.

In the experiment, participants were asked to group knowledge (i.e., notes and images) into concepts and create links between knowledge and concepts. Moreover, participants were prompted to group either by function, structure, or behavior (see Sec. 3.2.1 for more details).

On average, designers used 60% of the total notes (SD = 11.5). Based on single-factor analysis of variance (ANOVA), there was no significant difference in the number of notes used by participants with different prompts, showing little effect of the prompt on the number of notes used.

Notes’ FBS type, as described in Sec. 3.3, had little effect on their use frequency. On average, designers used 63% of behavior notes, 60% of structure notes, and 55% of function notes. Designers given the behavior prompt used 10% more behavior notes than those given other prompts, but this difference was not statistically significant.

Behavior notes were over-represented among the most-used notes. The 52 notes provided to participants consisted of 11 function, 14 behavior, and 27 structure notes. Of the top six notes used most by participants, half are behavior, despite relatively fewer behavior notes overall (see Table 5). These popular behavior notes were used by nearly all designers. Two were about the overall product design, two were about the hinge, and two were about the battery, indicating the high priority of such behavior knowledge to designers.

These findings point toward the importance given to product behavior by designers when describing the product. Interestingly, the behavior notes themselves were diverse in the aspects of the product they were describing—the hinge mechanism, holistic design, and battery—suggesting that designers did not fixate on one particular behavior of the product that might have been specifically promoted. However, as data on specific marketing messaging, or designers’ exposure to it, were not collected, this remains a question related to these results. Definition and extraction of product behavior should be an important focus of future work hoping to automate extraction from teardowns.

In the study, participants were asked to group knowledge into similar concepts. Moreover, concepts could also include other concepts, thus allowing participants to create nested or consolidated concepts.

Designers created six concepts (SD = 2.0) on average to group together knowledge. Designers from 9 of 23 studies created nested concepts, 3 from each FBS grouping prompt. Among the studies with nested concepts, designers given the behavior prompt on average had three nested concepts (SD = 2.5), those given the structure prompt had four (SD = 2.5), and those given the function prompt had five (SD = 2.0). However, these differences were not found to be statistically significant, given our sample size.

Overall, designers grouped together knowledge into 85 concepts categorized as structure, 48 concepts under behavior, and 39 concepts under function type. This shows the tendency for designers to group knowledge relative to their physical structure on the product, as opposed to the organization of knowledge notes, where behavior was largely over-represented.

For learning purposes, designers find the activity of grouping knowledge into concepts more helpful than linking or rating. We sought to determine if designers found a significant difference in participants’ perceived usefulness of the three core tasks of the study—grouping, linking, and rating for learning and knowledge transfer. We conducted a one-way ANOVA test with a single factor (usefulness for learning) and three levels (the tasks). We found that a significant difference in perceived usefulness between the three tasks existed (p < 0.05, N = 23, F = 8.04). Post-hoc comparisons using Tukey’s HSD test revealed significant differences between grouping into concepts and linking (p < 0.05) and rating (p < 0.01), revealing that grouping is more impactful for learning than other actions. In the survey, designers rated grouping equally important for learning (4.5, SD = 0.6) and knowledge transfer (4.4, SD = 0.7), on average.

These findings suggest that designers view different teardown activities to be useful for different purposes. Designers find grouping helpful in identifying commonalities in design and discovering design intents of individual components. Grouping also leads designers to consider how different aspects of the design inform and affect each other. The survey rating indicates that grouping is the most helpful activity if the goal is to learn from a teardown. Moreover, designers reported that in a professional environment with multiple teams involved, grouping could help identify relevant design goals for each team, while group nesting could help propagate design tasks across teams. This suggests that the activity of grouping of teardown knowledge could play an important role in knowledge transfer between teams.

These findings indicate that, during teardowns, activities around grouping of design knowledge, rather than linking or rating, are most effective for personal learning, as well as transferring knowledge to others. These results indicate that groups and group names would be a rich source of design knowledge for future work investigations of teardowns.

Participants were asked to link both knowledge and concepts to each other, to represent existing relationships between entities. Designers created six knowledge links (SD = 6.3) and six concept links (SD = 2.8) on average. Different FBS grouping prompts did not affect the number of links participants used. Three participants used more than 15 note links; this was attributed to their linking of images and notes to illustrate their relationships, which was not done by other participants.

A diversity of links evidence a variety of ways designers organize knowledge in design tasks. Each neighborhood of nodes represents links from multiple studies. Figure 3 shows two such neighborhoods around “cost” and “manufacturing.” The groups connected to “cost” are from five different studies, and two connections are made multiple times by different designers. The groups connected to “manufacturing” are from seven studies, and four connections are made multiple times by different designers.

The varying connections and relatively low overlap suggest that designers form relationships between groups of product knowledge differently, and that there may be value in sourcing knowledge organization from larger numbers of designers to reveal unexpected ways of relating knowledge. The observed diversity in knowledge organization might be an indicator of the diverse backgrounds of the participants, as different expertise may cause varying information to be linked together. If true, this indicates that varied backgrounds in generating product knowledge can lead to valuable, diverse information. This extends to current knowledge suggesting that crowdsourcing could be a valuable way to gather unique insights on designers’ approaches to knowledge organization, not just knowledge itself.

Designers primarily use links to describe design tradeoffs, requirements, and team collaboration. We observed three distinct reasons why designers developed links between groups of information (Table 3). First, designers made links to represent design tradeoffs. Participants used specific language to express tradeoffs, such as comparative adjectives (e.g., cheaper, more) and signs (e.g., >, =). Second, designers used links to identify design requirements, particularly regarding functional or structural aspects. Specific language that signified design requirements links included verbs for the name of the link. Finally, participants also used links to represent opportunities for team collaboration. During these studies, we observed these participants discussing how their knowledge groups map to different teams within their organization, and how frequent communication between these teams may help the design project. Perhaps of note is that the three designers in question held management roles within their professional organization (e.g., company founders and project managers).

These findings highlight the diverse roles that relational knowledge, captured in links, can play in establishing a designer’s knowledge space. Designers’ use of language is particularly important in distinguishing types of knowledge in links. Designers differentially use adjectives, verbs, and explicit references to organizational teams to indicate tradeoffs, requirements, and collaboration, respectively. This suggests that language can be a useful mechanism to uncover insights about participants’ knowledge organizing behavior. Furthermore, specific types of language could be a crucial facet in developing realistic design knowledge networks, and warrant further investigation.

Function notes were the least linked. Among the 67 note links designers created across all studies, only 13 involved function notes (see Fig. 4(a)), none of which linked one function note to another. As can be seen in the links, both other note types are equally represented. The lack of note links to and from function could suggest (1) an implicit prior understanding of function-behavior relationships describing the purpose of the product, allowing them to remain undocumented; (2) design tradeoffs, a key observed purpose of links, do not involve function and happen instead between behavior and structure; and (3) designers might prefer using group links to illustrate functional requirements, as discussed in Sec. 4.1.3.

Designers find linking more helpful for knowledge transfer than for learning. Participants rated linking as more valuable for knowledge transfer (4.2, SD = 0.8) than for learning (3.8, SD = 0.9). However, single-factor ANOVA (p = 0.16, N = 44, F = 2) did not reveal a significant difference. This suggests that the activity of searching for relationships between groups is conducive to thinking at a higher level about disparate aspects of the design. The selected quotes (Table 4) suggest that finding links between related notes, images, and groups can help uncover tradeoffs between different design goals, which in turn supports knowledge transfer between teams in professional environments. However, as no statistical significance was found in the ratings, these findings warrant further investigations.

In our survey, participants commented that the activity of note organization helped them learn about the product, as well as transfer this knowledge to others. On a 1–5 scale, designers rated the overall organizing activity 4.2 (SD = 0.5) for learning, and 4.5 (SD = 0.7) for knowledge transfer. All designers found that organizing knowledge helped enhance their understanding of the product. Several designers commented that different team environments may have varying dynamics and yield different outcomes for the organization task. In regards to knowledge transfer, several designers expressed that access to physical components or components in virtual reality environments would improve knowledge transfer. However, for learning, all designers found photos sufficient for transferring knowledge to others.

After collecting and analyzing data on knowledge organization in product teardowns, we sought to create a structured tool to navigate this information. This section presents and interprets the technique used to transform the knowledge organization data gathered through the teardown activity into a KG. We explore the properties and capabilities of the graph in order to showcase its abilities as a tool for organizing, learning, and transferring knowledge.

In order to situate the teardown data in a KG, we propose the schema in Fig. 5.

This identifies the four node types found in the KG: knowledge, media, consolidated concepts, and concepts. Knowledge and media both exist as raw data generated during the product teardown activity. Note that media nodes contain images and illustrations from the teardown activity. The consolidated concepts, concepts, and edge types were established by a separate set of participants in the knowledge organization portion of this study (Sec. 3.2). Concepts are groups of knowledge, media, and/or other concepts. Consolidated concepts are high-level groupings of concepts. Examples of each node type appear in Fig. 5, with further examples listed here: consolidated concept—“Housing,” concept—“Enclosure Design,” and knowledge—“One piece effective bi-stable hinge design.” Additionally, the two edge types are identified: relates and belongs. Relates refers to the direct linking between two nodes, while belongs refers to the nesting of one node under another, following ontology proposed by Damen and Toh [46]. Edges categorized as belongs are directed, while edges categorized as relates are undirected. Figure 5 identifies the relationships that exist within the KG and visualizes the hierarchical relationship between the node types.

The KG combines the experiential nature of the teardown study with ontology-based data coding and consolidation. These features manifest themselves as properties of the graph. Table 6 shows the properties of each node type. The product property associates the node to the specific product it refers to, while name and description are used as individual node identifiers. FBS is used to encode specific nodes according to this ontology. Role refers to the job and years of experience that the participant who encoded a specific node held. Table 7 shows the properties of each edge type. Description appears as a label explaining the reason why two nodes were connected by participants using the relates link. The edges also identify the role and years of experience of the participant who created them. The strength property refers to a participant-rated strength value of the connection between its two nodes. Finally, the type property refers to the consolidated code assigned to the edge, summarized in Table 3. Here, we have transformed our data into an accessible and scalable representation, with properties that enable insightful graph traversal, as given in the next section.

Navigating the graph with a specific query in mind is enabled by the intentional search features of the dataset. These provide results to directed questions that may arise from product teardowns or experiential activities as a whole, which many designers use as tools to discover and understand existing products’ designs. In this section, we propose several envisioned search queries and user interactions that are driven by the graph properties and which leverage the experiential nature of the graph, and then situate them in extended examples (Sec. 5).

Shortest path. In order to learn more about how our data are connected, we can choose two nodes of interest and view the shortest path between them. This pattern unearths the elements connecting pieces of interest and can provide clarity in the context of a product teardown. This is illustrated in Fig. 6, where we explore how the idea of the smart glasses being lightweight connects to the important battery component. Through this path, we learn about the relevance of the battery size to the lightweight aspect of the glasses and gain awareness of the design choices made for the user.

For example, 16% of shortest paths between all concepts pass through cost. Features, housing, cost, and manufacturing have the highest betweenness centrality among all nodes. These highly connected neighborhoods are shown in Fig. 3, highlighting important aspects for successful smart glasses products.

The number of central nodes in the graph suggests that the participants might have various opinions regarding the most important aspects of the product. This could be an indicator of the diversity in backgrounds of the participants, ranging from technical domains like manufacturing and electronics, to managerial roles. This finding also points to the wide range of design knowledge that is possible to collect from teardowns, an important aspect to consider as future work explores how to leverage design knowledge from teardowns to inform data-driven design approaches, e.g., machine learning models.

Filter by node and edge properties. The graph can be navigated by the various properties of the nodes and edges. First, navigating the connections between nodes to find links classified as tradeoffs or dependencies highlights where important design decisions and sacrifices must be made. The ability to search the graph by link type helps users learn more about design intent and why certain features are designed that way relative to neighboring features. Next, the graph can be filtered to see nodes tagged under a specific FBS heading, in order to find the important functions, structures, or behaviors of the product. Finally, the graph can be filtered for nodes appearing by participant role (manager, engineer, etc.) or by participant experience (in years) in order to learn how diversity in participants brings new paths and connections to the graph.

Measure strength of connections. Finally, the strength of connections within the graph can be used in order to find and rank the most important paths in the knowledge base. This metric can be used by considering the participant-ranked relationship strengths, which provides insights into connections the participants found particularly significant. Additionally, the strength metric can be calculated through either number of unique paths between a start and end node or by number of unique contributors to a particular path or subsection of the graph. Both of these metrics are indicators that those particular connections were consistently produced among the variety of participants.

Given the various node properties of the graph, we envision this graph as a tool to execute queries in order to filter through the data and find specific insights. In this section, we present examples of potential use cases that would leverage this KG. We present these queries as modular searches, which can be combined to answer complex questions regarding the teardown data. Queries for both intentional and exploratory searches are presented, allowing graph users to find both answers to specific questions as well as use the graph as a tool to explore the teardown data meaningfully. These queries serve to highlight possible ways in which the graph may be used through digestible examples.

By combining many of the envisioned search queries above, we look to enable users to learn through intentional search, navigating the graph to answer specific queries. We envision that the KG can serve many purposes to designers, from enabling analogical inspiration to gauging existing products. Here, we explore an example centered around intentional search, showcasing many of the insights and abilities our KG holds for benchmarking an existing product. Because of the “smart” nature of the product dissected in the teardown, we can infer that an important feature will be the battery that runs the specialized features of the glasses.

We begin by looking at the question “Which elements are dependent on the battery?”, a query a designer might be interested in learning more about. By querying for battery and finding its related concepts, we identify groups that are directly relevant to the high-level idea of the battery. Next, we explore a level of concepts further away, filtering for relations which are classified as dependencies or tradeoffs. Now, we’ve identified the specific concepts that are considered dependencies or tradeoffs with the battery, and want to abstract one level further, in order to get a clearer understanding of which high-level consolidated concepts answer our initial query. Navigating the graph shows us the final elements that are dependent on the battery functioning and the result to our query: Audio, Logic board, Cost, Housing, Weight, Right assembly (Fig. 7). These elements are represented as nodes in Fig. 7, appearing as the outermost light blue nodes.

We rate the strength of our results using the metric of number of unique paths. Here, we count how many unique paths exist between a start and end node, with a higher number of paths indicating more participants created relationships between these nodes. In this case, our start node is always battery and our end nodes are each of the six results mentioned above. As shown in Fig. 7, the audio node is ranked as the strongest result with four unique paths between it and the battery. These four pathways are highlighted in green to show their prominence in the overall results. Users can explore the pathways between the two nodes to learn more low-level details of the importance of this connection.

Finally, there are eight unique contributors to this overall resulting graph across five unique roles, exemplifying the diversity of participants involved in this knowledge organization process. This suggests the importance of curating knowledge from different backgrounds to account for varying design priorities, capturing both very common threads as well as unique perspectives.

Apart from very directed queries, the graph also supports undirected exploration. This lends itself to users spontaneously finding insights and supports the open-ended, fact-finding nature of product dissections. In this section, we propose a few potential pattern structures that hold interesting design implications, highlighting the KG’s ability to help drive innovation. We focus on presenting examples around design tradeoffs given the topic’s high importance in design decision-making [80].

Collaboration/Tradeoff patterns with FBS grouping. We can discover tradeoffs or collaboration patterns that are tied to FBS grouping, allowing for additional patterns that expose critical decision-making areas and pathways. For example, given that pathway A is a tradeoff of B and B is reliant on C, we infer that A is reliant on C and establish a new important link. An example is shown in Fig. 8, where the behavior-classified concept of Works-Like (left) is a tradeoff of the structure-classified concept of Assembly/Manufacturing of a Prototype (middle). The middle node is reliant on Engineering Validation Tests (right) and therefore creates a new inferred collaboration link between the outer two nodes, generating new knowledge from the graph by emphasizing the need to have operational “works-like” models in order to complete the validation tests run on these prototypes. The detailed KG also contains descriptions for links, providing another level of depth to the rationale behind these connections. Exploring these various relationship types among specific FBS types can expand to longer pattern pathways and drives insights into understanding design intent.

Structure-behavior tradeoffs. In this pattern, we wish to utilize the FBS ontology and link classifications to learn more about these relationships. In this example, we begin with a desired behavior we wish to achieve in the glasses (the quality of them being lightweight) and want to learn which structures are important toward achieving this behavior. We search for a structure-behavior tradeoff pattern linked to our desired behavior node and learn that the right temple structure is important (Fig. 9), with the graph providing an image of the temple area directly from the teardown activity. In Fig. 9, we see the detailed pathway and nodes connecting our desired behavior and its supporting structure, and have the ability to abstract a level to the concept level to learn about the higher-level structure that is playing a role in this behavior, in this case, the glasses frame. This pattern opens up the graph to explore these relevant areas and provides link descriptions to justify why this pattern appears.

Our results indicate three insights into how designers organize and acquire knowledge from product teardowns. First, we observe that while designers find grouping data to be more effective for learning, linking proved more helpful for knowledge transfer. Second, we find that designers employ links between data much more frequently than they do nests, and that links primarily serve to identify tradeoffs, requirements, and opportunities for team collaboration. Finally, a graph analysis reflects the diversity of perspectives on knowledge organization emergent in a constrained teardown activity.

The findings from our proposed interaction modes suggest that the intentional search capabilities of this KG can support knowledge retrieval from a design organization activity, and have several implications for organization and management of engineering design knowledge. First, at the level of an individual designer, specific to Wallace et al.’s concept of product knowledge [81] and to product teardowns especially, intentional search of our KG can help designers uncover design intent and understand a product’s affordances.

Second, also at the level of an individual designer, intentional search as illustrated here can help navigate the inherent tensions in levels of information that Damen and Toh identified in the engineering design process [82]. Intentional queries could also allow designers to more easily navigate tensions related to levels of effectuation of information (effectuation and causation), by allowing them to rapidly iterate on various modes of understanding an existing knowledge base, rather than awaiting acquisition of new knowledge to support their goal.

Lastly, at the level of organizations, the intentional search presented here offers a way for organizations to connect open-ended, nonspecific queries with detailed engineering design knowledge related to both product design and design process. Particularly intriguing is the possibility for KGs like ours to share implicit or tacit knowledge across organizations beyond specific engineering design contexts. This builds upon the capabilities of current semantic networks built for engineering design, which are well-supplied with explicit, easily transferable information, but lack this critical element of detailed, implicit knowledge. Additionally, our KG’s descriptive and accessible properties add a dimension not commonly seen in the current KG space.

Many implications of intentional search described earlier apply to exploratory search as well: enabling more effective discovery of product architecture and affordance during teardowns; helping designers navigate tensions between levels of information; and more readily availing design-specific and implicit or tacit knowledge across an organization. There are several implications specific to exploratory search capabilities of this KG, however. First, extending on previous findings by Li et al. that suggest KGs can be used to drive novelty in product design [58]. In particular, navigating the graph using the proposed patterns can aid in clarifying design intent and identifying critical decision-making areas. This connection highlights the support that KGs provide in using teardown information to identify areas of value and opportunity in engineering design. Second, exploratory search via the proposed “shortest path” pattern can particularly afford a balance between effectuation of knowledge and causation [82] by having a designer articulate an open-ended question (causation), then assembling existing knowledge in the most efficient manner (effectuation) to address it. Finally, while many other engineering KGs allow for exploratory search of their data, the nature of our teardown KG builds upon these KGs by facilitating clarity of connections between multiple nodes, and incorporating visualizations and media that support the descriptive nature of the graph.

Our findings provide a foundation for several directions for future inquiry at the nexus of knowledge organization, design theory, and data-driven design. First, we can conduct user studies to further explore the modes designers interact with knowledge organizations. Exploration of other knowledge-intensive design activities besides teardowns—e.g., product life cycle management or user research synthesis—would help expand upon and validate our insights into knowledge organization.

Second, the current data collection process can be time consuming, which restricts the number of insights and products included in the graph. Future work might leverage the current findings as a guide for collecting specific design knowledge data from other sources, such as patent databases, Wikipedia, online repositories of design challenges, or at the time of creation in computer-aided design and other design tools. Leveraging the underlying explicit relationships contained in existing semantic KGs might help automate the construction of the graph schema, as well as support unsupervised classification of knowledge, concepts, and links with NLP. This would allow for a more scalable, generalizable, and user-centered KG, in support of knowledge-based design methods.

Lastly, automated collection of design data, on the order of 1000 products, might enable the use of graph neural networks for deriving latent feature representations of knowledge, concepts, and links. We look to expand the graph representation and leverage graph neural networks to learn how to present design knowledge to maximize learning by the designer, while supporting knowledge transfer to stakeholders in other domains. We see our work as an initial step toward this vision.

Additionally, there are several improvements that would make the approach presented in our work more scalable and generalizable and address its limitations. First, while we sought to preserve the language of the designers involved in the original teardown, the notes from the teardown could be made more generic and applicable to several products, to remove any bias from emotional connections to details such as the company name. Second, more than one product could be shown, providing a wider range of information for participants and generating more diverse knowledge organization data across multiple products; similarly, more participants could participate in the teardown of said products, rather than just three as in our study. Third, having a large enough sample of participants under different roles can allow us to study how roles affect knowledge organization. Finally, group consolidation could be done with the use of NLP for consistency, leading to more accurately consolidated groups for data analysis. Addressing the above limitations would help expand the work presented in this paper toward an automated knowledge organization tool. By taking in data from current and future design knowledge bases, our work points toward a tool which learns to organize sparse and biased design knowledge.

In this work, we collect data about the organization of design knowledge from 23 design professionals by giving them unstructured design knowledge from a product teardown and guiding them through a series of tasks to add structure—knowledge, concepts, and links. The data we capture is diverse: it includes both implicit and tacit information and captures different patterns of organization. To fully represent the richness of the data, we propose a new knowledge graph. The knowledge graph enables complex queries, and supports both intentional and exploratory search, which are explored as extended examples. These findings enable valuable insights into design intent and innovation, and how to structure and share knowledge in complex design activities and organizations.

Careful elucidation of the knowledge extraction and organization behavior of professional designers engaging in teardowns is requisite for the development of impactful design support tools. Because teardowns can uniquely afford designers’ learning through active experimentation and concrete experience, insights gleaned from studying the approach can be extrapolated to describe other knowledge generating and organizing activities. Similarly, a nuanced understanding of knowledge organization practices could inform not just better support tools, but more realistic knowledge bases in support of more effective data-driven design.

We would like to thank Autodesk Tech Center, Autodesk Research, and the Teardown Library for facilitating the data collection, especially Tyler Mincey, Gabrielle Patin, and Dillon Johnson. We also thank Hyunmin Cheong and Edward Burnell for advice and support on the project. This work is based on preliminary findings that appeared in the 2021 and 2022 proceedings of the International Design Engineering Technical Conferences (IDETC) [83,84].

There are no conflicts of interest.

The datasets generated and supporting the findings of this article are obtainable from the corresponding author upon reasonable request.