Special Issue: Large Language Models in Design and Manufacturing

The first set of papers explores how LLMs generate diverse and innovative design solutions. For instance, in the paper titled “Do Large Language Models Produce Diverse Design Concepts? A Comparative Study With Human-Crowdsourced Solutions,” Ma et al. investigate the effectiveness of LLMs, specifically GPT-4, in generating diverse design solutions compared to human-generated concepts. The authors experimented with different prompt engineering techniques and parameter settings, creating 4000 LLM-generated solutions across five distinct design topics. These were compared with 500 human-crowdsourced solutions using a variety of diversity metrics. Findings reveal that while LLMs can produce numerous design solutions, they fall short of matching human diversity levels. Logistic regression analysis further indicated a semantic gap in some topics between human and LLM solutions. The study provides insights into LLMs' limitations in concept diversity, and contributes valuable guidance for future research on enhancing LLM-generated design creativity.

Chen et al. in the paper titled “A Conceptual Design Method Based on Concept–Knowledge Theory and Large Language Models” enhance the concept–knowledge (C–K) theory for conceptual design by integrating LLMs to facilitate interdisciplinary knowledge retrieval and concept generation. The authors redefine the C and K spaces, categorizing concepts into function, appearance, and technology to create structured connections between concepts and relevant knowledge. LLMs are leveraged for dynamic knowledge retrieval and reasoning, supporting concept ideation, evaluation, and refinement. A case study on wearable devices demonstrates the method's effectiveness in generating innovative and feasible design concepts. This work shows how LLMs can overcome traditional knowledge limitations in conceptual design, and help designers access a broader knowledge base without specialized learning curves.

In the paper titled “Reconstruction and Generation of Porous Metamaterial Units Via Variational Graph Autoencoder and Large Language Model,” Khanghah, Wang, and Xu propose and compare two deep generative model-based approaches for the design representation, reconstruction, and generation of porous metamaterials. The first approach utilizes a dual decoder variational graph autoencoder for predicting both nodes and edges of the graphs representing the solid and pore phases in the porous structure. The second approach employs a variational graph autoencoder as the node generator and a fine-tuned LLM as the edge generator. A comparative study was carried out, in which the authors observed that LLM demonstrated significant strength in reconstructing the edges in graphs. However, the selection of an appropriate LLM is crucial for achieving high prediction accuracy. This research also demonstrates the zero-shot learning capability of the proposed model by generating structural patterns that were not included in the observational dataset.

This group of papers covers research on LLMs' capacity to understand and extract technical information. Doris et al. in the paper titled “DesignQA: A Multimodal Benchmark for Evaluating Large Language Models' Understanding of Engineering Documentation” introduce “DesignQA,” a specialized benchmark designed to assess multimodal large language models' (MLLMs) abilities to interpret complex engineering documentation. This benchmark integrates textual and visual information, including computer-aided design (CAD) models and engineering drawings, based on data from the Formula Society of Automotive Engineers (SAE) competition. DesignQA features tasks divided into rule extraction, comprehension, and compliance, reflecting real-world engineering challenges. The paper evaluates models such as GPT-4 and Claude-Opus, highlighting that, despite some successful instances, current MLLMs struggle with extracting specific rules and analyzing technical drawings. DesignQA aims to guide future improvements in artificial intelligence (AI) capabilities for engineering design, with the benchmark publicly available to encourage further research.

Li et al. in the paper titled “Integrating Graph Retrieval-Augmented Generation With Large Language Models for Supplier Discovery” introduce a novel approach that combines LLMs with knowledge graphs (KGs) to address the challenges of modern supply chain management, specifically in supplier discovery. The methodology leverages ontology-driven graph construction and thesaurus-supported retrieval-augmented generation to convert unstructured supplier data into a cohesive KG, making it easier to locate and assess suppliers. By building a supply chain knowledge graph with multiple entity and relationship types, the system supports an advanced question answering system that delivers high-quality, interpretable answers. The framework is designed for scalability and adaptability, enabling it to incorporate new data seamlessly and apply across diverse industries. This approach not only improves the visibility of small and medium-sized manufacturers but also enhances agility and provides strategic insights in supply chain decision-making.

In the paper titled “Design Knowledge as Attention Emphasizer in Large Language Model-Based Sentiment Analysis,” Han and Moghaddam introduce a new method to incorporate design knowledge into LLMs for aspect-based sentiment analysis (ABSA). Addressing gaps in current ABSA methods, the authors propose a unified model called ACOSI (aspect, category, opinion, sentiment, implicit indicator) that simultaneously extracts explicit and implicit opinions, expanding beyond conventional sentiment tasks. A novel position encoding method emphasizes design knowledge in the model, enhancing its performance in processing implicit sentiments. Experiments using e-Commerce data highlight the model's scalability and the utility of domain-specific knowledge. The approach offers designers richer insights from user reviews, and advance early-stage product ideation.

In the paper titled “Evaluating Large Language Models for Material Selection,” Grandi et al. investigate the potential of LLMs to support material selection for conceptual design. Recognizing material selection as a complex, multi-criteria process affecting product performance, sustainability, and cost, this paper examines LLMs' ability to recommend materials comparable to human experts. Using a dataset of expert material preferences across various design scenarios, the researchers evaluated different LLMs, including GPT-4, Mixtral, and a fine-tuned model named MechGPT, employing prompt engineering techniques to improve model alignment with expert choices. The results reveal challenges in the LLMs' ability to match expert decisions due to biases and limited variance in recommendations. However, certain methods, such as parallel prompting, improved performance, suggesting the potential for more refined approaches in guiding LLMs toward expert-level material selection. The paper concludes with a call for further research to enhance LLMs' effectiveness and reliability in design-oriented applications.

Ataei et al. in the paper titled “Elicitron: A Large Language Model Agent-Based Simulation Framework for Design Requirements Elicitation” present a framework called Elicitron that utilizes LLMs to automate and improve requirements elicitation in product design. Elicitron employs LLM agents to simulate a wide range of user personas, enabling it to explore diverse and latent needs that traditional methods often overlook. It generates simulated user interactions to identify pain points, and then conducts interviews to uncover latent needs. The authors report on three experiments demonstrating that Elicitron produces a diverse set of user needs, resembles empathic lead-user interviews effectively, and can identify more latent needs than traditional approaches. The framework shows promise in streamlining early-stage product development, reducing costs, and fostering innovation by automatically identifying user requirements using LLMs.

Li, Sun, and Sha in the paper titled “LLM4CAD: Multimodal Large Language Models for Three-Dimensional Computer-Aided Design Generation” present research to answer two questions: (1) To what extent can multimodal LLMs generate 3D design objects when employing different design modalities or a combination of various modalities? (2) What strategies can be developed to enhance the ability of multimodal LLMs to create 3D design objects? An approach, called LLM4CAD, was developed to enable multimodal LLMs in 3D CAD generation. Experiments were performed to evaluate the efficacy of GPT-4 and GPT-4V models as part of the LLM4CAD with different input modalities, including the text-only, text + sketch, text + image, and text + sketch + image data. The authors found that both GPT-4 and GPT-4V showed significant potential in the generation of 3D CAD models by just leveraging their zero-shot learning ability. In the experiment of GPT-4V, four input modes were tested and GPT-4V showed superior performance with text-only input. However, when examining category-specific results of mechanical components, multimodal inputs start to gain prominence with more complex geometries.

In the paper titled “Transformer Based Offline Printing Strategy Design for Large Format Additive Manufacturing,” Xie et al. propose a new approach to optimize printing strategies for large form additive manufacturing through the development and application of a transformer-based model focused on the dynamic prediction and management of temperature profiles across the print surface. The You Only Look Once (YOLO) algorithm is adopted to track the extruder's position in real-time during the printing process. Then, a transformer-based model for temperature prediction and an optimization model for layer time control to determine the printing strategy are developed. The transformer model is able to generate temperature profile predictions at all positions within a layer simultaneously, and for various geometries. The optimization model significantly reduces layer time. The experiments have demonstrated that the transformer architecture can effectively capture the complex spatiotemporal relationships at different positions on the surface of the printed object. This allows for a more nuanced understanding of the thermal dynamics during printing, leading to more precise control over the manufacturing process.

In the paper titled “Putting the Ghost in the Machine: Emulating Cognitive Style in Large Language Models,” Agarwal, Jablokow, and McComb evaluate the efficacy of LLMs to emulate aspects of Kirton's adaption-innovation theory, which characterizes individual preferences in problem-solving. LLMs were used to generate solutions for three design problems using two different cognitive style prompts: adaptively framed and innovatively framed. Solutions generated by the LLMs were evaluated with respect to feasibility and paradigm relatedness, which are known to have discriminative value in other studies of cognitive style. We found that solutions generated using the adaptive prompt tend to display higher feasibility and are paradigm preserving, while solutions generated using the innovative prompts were more paradigm modifying. This paper demonstrated that LLMs can be prompted to accurately emulate cognitive style.

Jiang, Huang, and Shen in the paper titled “Generative Artificial Intelligence-Enabled Conceptualization: Charting ChatGPT's Impacts on Sustainable Service Design Thinking With Network-Based Cognitive Maps” examine the impacts of generative AI and ChatGPT on design thinking process and outcomes through controlled experiments with novice designers. The authors carried out experiment to visualize the design thinking by network-based cognitive maps, then evaluate design outcomes and systematically analyze characteristics of design thinking development under different tool interventions. The results indicate that ChatGPT enhances design concept novelty systematically but has limited impact on originality and sustainability. It also found that ChatGPT plays an active role in fostering thinking divergence and fluency, especially in providing relevant guides for developed ideas and accelerating the evaluation and creation process. The network-based cognitive maps reveal distinct shifts and styles influenced by ChatGPT, providing references for novice designers using such tools to enhance inspiration and design fluency, and also effectively employ diverse tools during specific concept generation stage.

Zhang, Zhao and El Haddad in the paper titled “Understanding the Impact of Applying Large Language Model in Engineering Design Education” carried out research to evaluate the impact of LLMs specifically the ChatGPT3.5 on mechanical engineering design education focusing on concept generation and detailed design modeling stages. They found that LLMs such as ChatGPT significantly broadens concept generation diversity but also introduces bias for existing popular designs. Furthermore, LLMs can support detailed design by suggesting functions for CAD modeling. However, its textual nature and the occurrence of unreliable responses limit its usefulness in detailed CAD modeling tasks, highlighting the irreplaceable value of traditional learning materials and hands-on practice.

The collection of articles in this special issue sheds light on the transformative potential and current limitations of LLMs in the fields of engineering design and manufacturing. LLMs have demonstrated remarkable capabilities in automating ideation and knowledge extraction, and the studies included offer insights into diverse applications from concept generation to design specification and automation. However, challenges remain regarding LLMs' ability to handle complex, domain-specific knowledge and their integration into existing design and manufacturing workflows.

This special issue groups the contributions into four thematic areas: ideation and concept generation, knowledge and information extraction, design specification and automation, and cognitive and educational impacts. By categorizing the papers in this way, we gain a clearer view of the multifaceted roles that LLMs can play across various stages and activities of the engineering process. The articles explore practical applications as well as methodological advancements, presenting both theoretical frameworks and experimental validations. These studies underscore the importance of prompt engineering, retrieval-augmented generation techniques, and multimodal data integration in enhancing LLM performance for engineering-specific contexts.

Overall, this special issue demonstrates the promise of LLMs in advancing design and manufacturing practices, while also highlighting key areas for future research. As LLMs continue to evolve, their integration into complex engineering environments will require ongoing innovation, cross-disciplinary collaboration, and careful evaluation to realize their full potential in transforming these industries. We hope this special issue serves as a foundation for further exploration and inspires future developments in LLM applications within engineering design and manufacturing.