A Novel Diagnostic Tool for Human-Centric Quality Monitoring in Human–Robot Collaboration Manufacturing

Mass production has long been the standard in manufacturing, allowing efficient use of available resources and a corresponding reduction in production costs. In recent years, however, there has been a shift toward mass customization, an approach to production that allows products to be individually tailored to meet each customer's specific needs and preferences [1]. Several factors have driven this shift, including technological advances, increased demand for customized products, and growing awareness of mass production's environmental and social impacts [2]. As a result, manufacturers are increasingly turning to mass customization as a way to remain competitive in the marketplace and meet the evolving needs of their customers, thus representing a significant paradigm shift in how goods are produced and consumed.

While greater product variety can increase market share and sales volumes, it also increases product complexity and costs [1]. As a result, mass customization requires a flexible production system that can adapt to product volume and type variations. An effective approach to mass customization is the use of collaborative robots (also called cobots) in what is known as Human–Robot Collaboration (HRC) [3]. This approach combines the flexibility and versatility of human operators with the precision of cobots, creating a flexible system capable of assembling different product variants in the same workstation [4].

Interest in HRC has grown with the development of Industry 4.0 and is becoming increasingly important with the emergence of Industry 5.0. Indeed, the main goal of Industry 5.0 is to put human well-being at the center of production systems in order to provide sustainable prosperity for long-term development [5–7]. The technologies and processes used in Industry 5.0 are designed to enhance the capabilities of human operators, as in the case of cobots. This approach departs from traditional production methods, which prioritize efficiency and automation over the satisfaction of human workers. This shift toward human-centered production should lead to improved productivity, greater job satisfaction, and a more sustainable manufacturing sector toward mass customization.

While the benefits of HRC are clear, there is a lack of comprehensive tools that can assess and monitor the quality and well-being aspects of these systems. Therefore, a need exists for an evaluation tool that not only considers production quality but also considers the well-being of the human operator. With the growing emphasis on human-centered production in Industry 5.0, such an assessment tool becomes increasingly crucial.

To address the lack of comprehensive tools for assessing and monitoring HRC systems taking into account both process performance and human-centered performance for highly customized product variants, this research aims to answer the following research question: How can a diagnostic tool be developed to assess the performance of HRC systems considering the specificities of highly customized product variants?

In response to this research question, this paper introduces a novel tool, the “Human–Robot Collaboration Quality and Well-Being Assessment Tool” (HRC-QWAT). This tool integrates two indicators, a product and process quality indicator and a human stress response indicator, to assess and monitor the quality of an HRC system, specifically tailored to accommodate the unique challenges and variations associated with highly customized product variants. Unlike previous studies that focused on individual measurements and indicators, the HRC-QWAT incorporates multiple dimensions of evaluation, encompassing product and process quality as well as human well-being. This comprehensive approach enables a holistic assessment of the collaborative assembly process, capturing the interplay between quality outcomes and the well-being of human operators.

The versatility and adaptability of the HRC-QWAT are demonstrated by its applicability to single-variant and small-batch variant production scenarios, catering to diverse production contexts. Thus, the HRC-QWAT fills a critical gap in assessing and monitoring HRC systems, providing practitioners and researchers with a valuable tool for evaluating, diagnosing, and optimizing HRC systems in the context of diverse and customized product variants.

To show the practical implementation of the HRC-QWAT, a real-life case study was conducted involving the assembly of electronic board variants using a human–robot collaborative assembly system. The methodology consists of two main phases: (1) the realization phase, in which the HRC-QWAT is constructed by collecting historical experimental data and developing a model that relates the two performance measures, i.e., total defects (cobot-related and human-related errors) and human stress response, to represent the overall quality of the system; and (2) the use phase, in which the HRC-QWAT is used as a reference for predicting future products/batches and identifying critical products in terms of defects and human stress response. The HRC-QWAT can be used to identify critical production scenarios and implement necessary corrective actions to maintain the desired quality level while taking into account the well-being of human operators, thus advancing human-centered production practices within the framework of Industry 5.0.

The remaining paper is structured as follows. Section 2 summarizes the main studies in the field of quality in HRC. Section 3 presents the HRC assembly system used as a case study. Section 4 illustrates the complexity assessment of the product HRC assemblies. Section 5 presents the data collected on product and process quality and their relationship with assembly complexity. Section 6 illustrates data on human well-being and discusses the relationship with assembly complexity of product variants. Section 7 presents the novel diagnostic tool called HRC-QWAT, which shows the potential for single-variant and small-batch production. Finally, Sec. 8 concludes the paper.

HRC is a rapidly developing field with promising applications in service, social, and industrial contexts. When designing and implementing an HRC system, evaluating its quality is crucial to ensure that the system meets individual, collective, and production needs or objectives. From an engineering point of view, quality refers to the degree to which a system, product, service, or process conforms to specified requirements and conditions [6]. Quality models, such as conceptual or definition models, are commonly used in engineering to support, evaluate, and manage quality [8]. These models outline a set of quality attributes and their potential interrelationships and serve as a guide for selecting relevant factors for experimental validation of applications, services, or systems.

However, efforts to identify and classify factors, measures, and metrics that describe quality in the HRC field are still rare, especially from a human-centered perspective [6,9]. The industrial perspective can be categorized into two interests: performance-centered and human-centered. The former considers robots as a means to optimize the production process, often involving full automation and substituting human workers with machines, while the latter aims to improve human well-being by respecting their roles, needs, jobs, talents, and rights [10–12]. Consequently, there is a trade-off between optimizing the production process and optimizing the well-being of the operators, which requires the use of performance measures specific to the collaborative environment.

According to Ref. [6], performance measures for HRC are variables that can be obtained from physical measurements or an aggregate of facts to assess the current or final state of the human, robot, process, or interaction. These measures can be grouped into four categories:

1. Time behavior measures indicate the response and processing times required to perform functions or complete tasks.

2. Process measures are an aggregation of facts related to task completion, workspace design, safety, or product quality.

3. Physiological measures are obtained from body measures, such as heart rate, to understand the current state of the human.

4. Human–robot physical measures are obtained from sensors that indicate the current state of the interaction, such as the distance between the human and the robot.

Moreover, performance metrics for HRC can be defined as a combination of direct measures used to express a rate, average, or input/output relationship [6]. Efficiency and effectiveness are considered the main attributes used to evaluate such performance. Efficiency metrics assess the use of resources, i.e., the input/output ratio. On the other hand, effectiveness metrics assess the accuracy and completeness of the achievement of specific objectives, measuring the relationship between actual and expected results. These metrics assess whether HRC systems are “doing things right” and “doing the right things,” respectively.

As far as the human-centered perspective is concerned, quality factors that have received more attention in the robotics literature are safety [13], trust [14], attitudes and acceptance [15], mental and physical workload [16,17], situation awareness and mental models [18,19], emotional responses [20,21], and anxiety [22].

Additionally, the review paper [6] identifies seven emergent research topics that could have a significant impact on future Industry 5.0 applications, including (i) non-invasive monitoring and online analysis of human factors, (ii) individualized HRC, (iii) transparent robotic systems, (iv) fluency, (v) adaptive workload systems, (vi) privacy in data-driven HRC, and (vii) benchmarks.

Regarding point (i), the Industry 5.0 paradigm aims to optimize human well-being through human-centered smart environments. However, most tools for assessing human factors in HRC require offline or intrusive techniques. Creating accurate, non-invasive, and online ergonomic assessment tools that require short preparation represents a relevant challenge in HRC for manufacturing settings [23,24]. One of the most widely used tools for this purpose is the Digital Twin (DT), which allows the human comfort and flexibility of the cobot to be improved in a non-intrusive way [25]. Several DT applications have already been implemented in the area of collaborative assembly and disassembly [26].

Concerning point (ii), applications enabling collaborations between humans and robots are generally short and static for practical reasons [27]. However, individualized machine collaboration is essential for Industry 5.0. Nowadays, various technologies have been identified that enable machine collaboration, such as human action recognition, intention prediction, augmented, virtual or mixed reality, exoskeletons, and collaborative robots. Personalized HRI systems can continuously collect and process personal and physiological data, adapt to individuals’ needs and preferences, and maintain long-term interactions [27,28]. Hedonics factors, which mostly focus on individual goals, require more research attention on applications for Industry 5.0 [29]. Additionally, human-centered initiatives need to consider technologies enabling job satisfaction, work-life balance, and up-skilling and re-skilling of workers [12].

Regarding point (iii), Industry 4.0 applications use black-box AI to enhance autonomy, while Industry 5.0 requires transparent AI that interacts with humans. In HRC, this transparency includes predictability, legibility, and explainability. Legibility enables observers to quickly infer correct goals, while predictability matches expectations. Creating legible trajectories is a broad open issue. Multimodal systems for anticipating human actions face high-dimensional data, which dimensionality reduction techniques can address. AI aims to explain robot behavior to users and improve trust and situational awareness, but challenges include creating methods for generating explanations and evaluating their effectiveness [30,31].

With regard to point (iv), fluency is not considered a metric but rather a quality of interaction in HRC, as described in Ref. [32]. In the HRC environment, fluency refers to the seamless interaction between humans and robots. It involves a high level of coordination, well-synchronized joint activities with precise and efficient timing, and dynamic adaptation of plans and actions. However, fluency is still a relatively new concept in HRC research, and proposed metrics for fluency are often task-specific [32]. Recent studies, such as those of Hoffman [32], have categorized fluency metrics as subjective or objective. However, due to the somewhat vague and ephemeral nature of fluency, it remains a topic of discussion in the robotics community, with further research needed to fully understand the factors affecting fluency and to design metrics that can assess it in various collaborative settings.

Concerning point (v), real-time workload assessment algorithms using physiological measures, such as heart rate, electrodermal activity, and skin temperature, can accurately estimate optimal workload levels in humans [17,33]. This information can be used to improve task performance, reduce errors, and prevent accidents by changing interaction mediums, level of autonomy, and reallocating tasks and responsibilities between humans and robots [33]. Such systems are called adaptive workloads or adaptive teaming systems [17]. The use of these algorithms in various human–robot teaming scenarios remains an open challenge [34].

Regarding point (vi), data-driven technologies like big data, machine learning, cloud computing, and IoT can enhance production performance and human working conditions. However, Industry 4.0 has largely overlooked the human factor and the privacy issues arising from the collection, storage, and processing of personal data that these technologies entail [35,36]. In human-centered manufacturing, privacy efforts must focus on protecting workers’ personal information and ensuring data security [35]. Cybersecurity assessment criteria for HRI in automobile manufacturing have been proposed [37], but comprehensive metrics are needed for HRI and HRC [37].

Finally, with respect to point (vii), international robotics competitions have become a valuable tool for evaluating the performance of robotics systems, providing a form of reproducibility and enabling the evaluation of non-competing systems. Although the scoring mechanism tends to hide the underlying characteristics of the system, competitions allow systems to be compared by linking relevant metrics to the score and explaining which aspects influenced the score and in what way. Typically, the score is based on objective task completion (e.g., image classification accuracy), with few competitions evaluating safety in HRI. However, there is a shift toward more human-centered objective evaluations, exemplified by the safety score in the Future Convenience Store Challenge [38,39].

Based on the literature review, it is evident that many approaches proposed in the field of HRC have a performance-centered perspective, which fails to consider the full potential of HRC applications. Toward a human-centered society and industry, HRC researchers should broaden their perspective beyond mere task fulfillment and adopt holistic approaches that enable robotic systems to achieve both collective and individual goals. In line with this viewpoint and to help address the challenges identified in the seven emerging research topics, the “Human–Robot Collaboration Quality and Well-Being Assessment Tool” (HRC-QWAT) has been proposed. In detail, the proposed tool can respond to the challenges mentioned above as follows:

1. Non-invasive monitoring and online analysis of human factors: HRC-QWAT allows for real-time, non-invasive monitoring of human operators’ stress levels and well-being through the integration of wearable devices and sensors. This real-time evaluation ensures a prompt intervention to reduce stress levels, fostering a more efficient and balanced working environment.

2. Individualized HRC: The tool offers the possibility of individualized HRC by considering the unique physiological responses of each worker. This personalized approach promotes a more efficient and harmonious human–robot interaction, potentially leading to improved productivity and well-being.

3. Transparent robotic systems: Transparency is facilitated as the tool evaluates the collaborative process based on clear performance indicators and stress responses. These evaluations can be shared with human operators, fostering an understanding of the robot's function and promoting trust and collaboration.

4. Fluency: By measuring the quality of the collaboration through multiple performance indicators, the HRC-QWAT contributes to assessing the fluency of the human–robot interaction and collaboration. This analysis promotes the optimization of joint actions and the creation of more fluid and synchronized interactions.

5. Adaptive workload systems: The HRC-QWAT real-time monitoring of human stress responses can inform adaptive systems. By detecting stress or overwork, the system can automatically adjust the workload distribution between humans and robots, improving efficiency and reducing the risk of human error or health implications.

6. Privacy in data-driven HRC: While HRC-QWAT uses data-driven methods for evaluation, it is designed with the utmost respect for privacy standards. Personal and sensitive data are strictly used for the intended purpose of enhancing human–robot interaction and are safeguarded according to the highest security protocols.

7. Benchmarks: The proposed tool also serves as a benchmarking instrument for HRC in different scenarios. By providing comprehensive metrics on both the performance of the collaborative process and the well-being of the human operator, the HRC-QWAT offers a valuable standard against which different collaboration setups can be compared.

Accordingly, the HRC-QWAT serves as a comprehensive tool, specifically addressing the identified challenges in HRC, thereby offering a strategic instrument for human-centered Industry 5.0.

An experimental campaign is conducted to assemble six different variants of electronic boards (from variant V1 to variant V6) using the ARDUINO UNO starter kit (ARDUINO^®). The choice of electronic boards is based on the fact that, by using the same components, highly customized products with varying levels of complexity can be assembled (as will be discussed in Sec. 4). Moreover, these boards allow real-time verification of the correct functioning of the products, i.e., their proper assembly.

The ARDUINO UNO starter kit is composed of (i) the components that are assembled to make up the various boards listed in Table 1 (e.g., the jumper wires that carry current between the various components); (ii) the microcontroller, i.e., a small computer that enables the circuits to function; and (iii) the breadboard, i.e., a board on which the actual circuit can be built. The breadboard consists of rows and columns of holes that conduct electricity through thin metal connectors under the plastic screen, allowing the circuit components to be connected. The ARDUINO UNO Breadboard is defined as “seamless” as the components do not need to be welded but simply inserted into the holes. Figure 1(a) shows an example of an assembled electronic board (variant V3), while Fig. 1(b) displays the product circuit diagram.

Each of the selected products has a varying number of components, which are connected to the breadboard. As outlined in Sec. 4, the six electronic boards were chosen to span a broad range of assembly complexity. Table 1 indicates the type and number of components required for each of the six electronic board variants (V1–V6).

The assembly of the six electronic board variants was conducted using a Universal Robots™ UR3e cobot, as depicted in Fig. 2. The boards were assembled using an OnRobot™ RG6 gripper, a versatile gripper capable of handling small objects and a range of other objects. Six skilled operators, proficient in electronics and electrical engineering, were involved in the assembly process of all six electronic boards, following a random order to prevent any learning effects. During the preliminary stages, each operator underwent training sessions to ensure consistent proficiency among the participants and minimize the potential impact of varying skill levels on the results. These training sessions were carefully designed to familiarize the operators with the assembly process and equipment, allowing them to develop a solid understanding of the tasks involved in the HRC assembly. Table 2 provides an overview of the participants’ characteristics, including relevant information such as age, gender, domain knowledge of HRC, and domain knowledge of assembly tasks. The inclusion of skilled operators with expertise in electronics and electrical engineering helped ensure that the participants were familiar with the intricacies of electronic board assembly and could contribute effectively to the HRC trials.

In the assembly phase, the cobot handed over the required components to the operator, who assembled the electronic boards in a predetermined order, defined based on circuit theory [40]. The operator completely controlled the logistic tasks by activating the cobot using a button. After the assembly was completed, an experienced external operator (who was not involved in the assembly) conducted an offline quality control check to identify any defects in the final product. Data on overall assembly defects (cobot-related and human-related errors) were collected during the trials, as described in Sec. 5. Additionally, data on the operators’ stress response during the assembly phase were collected, as per Sec. 6.

In scientific literature, complexity is typically used as a metric to predict production performance, including production times and defects. Indeed, it is often found that a reduction in complexity is associated with a significant performance improvement [41–43]. In this study, the structural complexity model, first introduced by Sinha et al. [44] and later adapted by Alkan and Harrison [45] and Verna et al. [43,46], serves as the foundation for assessing the assembly complexity of selected ARDUINO products. This model, originally developed for manual and fully automated assembly, is extended to the HRC assembly of the present case study, where the robot primarily performs organizational and logistical tasks, such as selecting components to be assembled in a predetermined sequence and delivering them to the human assembler. Adapting and integrating the structural complexity model to the domain of HRC assembly for highly customized product variants represents an innovative aspect of this study. This enables a quantitative assessment of assembly complexity within the context of mass customization.

The six product variants were selected to cover a wide range of assembly complexity. In the case study, each hole on the breadboard was modeled as a single component. This assumption allows multiple connections between the components and the board to be modeled and distinguished from single connections. For example, pushbuttons, i.e., the components that close a circuit when pressed, consist of four different pins that need to be connected to the board. As this type of connection is more complex than connecting a single-pin component, it was necessary to model the individual holes on the board to distinguish these different cases.

Thus, the Lucas method provides a normalized assembly index that penalizes the physical attributes (e.g., component positioning and fastening, assembly direction, visibility, alignment, and resistance to insertion) that affect assembly difficulty.

Each entry in the AM matrix indicates the presence of an assembly connection between the component and the breadboard. As an example, Fig. 3 shows the AM matrix of product variant V3.

As shown in Table 4, the connection complexity of each component with the breadboard (c_pr) in the six electronic board variants (V1–V6) can take different values depending on multiple factors. For example, the connection complexity of long wires to the breadboard ranges from 3.7 to 6.3, depending on how the component is inserted into the breadboard and what other components are already connected. A complexity score of 5.3, for instance, is given if the wire needs to be bent to make the connection, and 6.3 if the connection is made with reduced visibility.

According to the increasing total assembly complexity C, Table 6 lists the complexities C₁, C₂, and C₃ of the selected product variants. It is worth noting that an increase in complexity does not always imply an increase in the number of components. In fact, although variant V5 has more components than variant V6, the total complexity of variant V6 is higher than that of variant V5. This is due to the different nature of the components that compose the different products, the nature of the connections, and the architecture of the final assembly.

In this section, complexity measures are integrated into the analysis of product and process quality, providing a novel perspective on the relationship between assembly complexity and the occurrence of defects in collaborative assembly processes for customized products.

During the manufacturing process, quality data on the overall defectiveness of the product and process were collected to assess the quality of the HRC system (see Table 7). Specifically, for each product variant assembly, the total number of defects (both in-process defects occurring during assembly—referred to as D₁—and offline defects detected during offline quality control—referred to as D₂) was recorded. A classification was made for both types of defects, D₁ and D₂ (see Table 8). During the manufacturing process, the assembly operators and the quality control operator filled out Table 8, indicating the number of defects found in each category for each assembled board. Certain defect categories, such as “Unpicked Component” and “Slipped Component,” specifically relate to errors made by the cobot during the assembly phase. These categories reflect instances where the cobot failed to pick up a component or where a component slipped during the cobot handling. It is important to highlight that these defect categories capture cobot-related errors occurring during the assembly phase. Furthermore, it should be noted that the defects recorded in the in-process and offline phases reflect a combination of both cobot-related and human-related errors. This means that the defect data collected encompasses the performance of both the cobot and the human operators involved in the assembly process. To achieve a holistic view of the quality of the system, the total number of assembly defects D_tot (i.e., the sum of in-process and offline defects) was considered and analyzed (see Table 7).

The exclusion rule used was the Modified Interquartile Range Method, which is widely recognized as a practical and effective method for identifying outliers, taking into account the sample size [49]. The relationship between the total number of defects recorded by the six operators for each of the six variants of electronic boards and the complexity of the assembly (calculated as described in Sec. 4) was then analyzed. The “operator factor” was not considered in the analysis after checking its non-significance at a 95% confidence level using a two-way ANOVA (p-value of 0.290). The Poisson regression model was used for the analysis, as total defects are count data [50]. The logarithm and square root link functions were considered, and different models were compared up to the third order of the predictor (i.e., assembly complexity C). The selection of the best model was made based on Akaike's Corrected Information Criterion (AICc) and Bayesian Information Criterion (BIC), goodness-of-fit tests (Deviance and Pearson tests), and deviance residual plots [50,51]. The Deviance and Pearson tests assessed whether the predicted number of events deviated from the observed number in a way that was not predicted by the Poisson distribution. If the p-value was less than the significance level, the null hypothesis that the Poisson distribution provided a good fit could be rejected [50,51].

Figure 4(a) shows the total defects recorded during the experiment and the predicted curve obtained by Poisson regression with 95% confidence and prediction intervals are represented. Moreover, Fig. 4(b) shows the deviance residual plots, where the residuals appear satisfactory overall. Also using the Anderson–Darling test, the hypothesis of normality of the residual distribution cannot be rejected at the 95% confidence level (p-value = 0.194, which is higher than the significance level of 5%). The results obtained for product and process quality show that the increase in assembly complexity of the variants leads to an increase in the total number of defects, following a non-linear trend.

In this section, existing methodologies for assessing human well-being are integrated and adapted to capture the impact of assembly complexity on the human stress response in the context of mass customization, showcasing the originality of the proposed approach.

Physiological measures can be used to assess the state of human well-being during production, providing an objective measure compared to self-report tools, which may suffer from retrospective post-task bias [52]. Electrodermal activity (EDA) data are used in this study as a measure of human well-being, as it is commonly used as an indicator of the human stress response [53]. The Empatica E4 wristband (see Fig. 5(a)), a non-invasive biosensor that records EDA information at 4 Hz, was used to collect the EDA data. In addition to EDA, the Empatica E4 also records information on pulsed blood volume (BVP), operator pulse motion (ACC), heart rate variability (HRV), and temperature (TMP). Figure 5(b) shows an example of the raw output provided by the Empatica E4.

For each test performed by the operators, this raw signal was recorded and then analyzed using the EDA Explorer software [54]. This software removes any external noise from the raw signal and decomposes the EDA signal into two types of signals: the tonic signal and the phasic signal. The tonic signal refers to the long-term fluctuations of the EDA signal that are not explicitly triggered by external stimuli. Changes in Skin Conductance Level (SCL) are the best indicator of tonic activity. On the other hand, phasic activity refers to transient changes in EDA that are triggered by typically perceived and externally delivered stimuli. It is best characterized by Skin Conductance Response (SCR) changes. Accordingly, the SCR can be defined as a change in the amplitude of the EDA signal from the SCL to a peak in the response [53].

This model was the best-fitting model compared to various linear and non-linear models, considering the goodness-of-fit statistics and residual analysis [55]. The statistical significance of the parameter estimate is confirmed by checking that the 95% confidence intervals for the parameters, calculated from the corresponding Standard Errors (SE) reported in Table 10, exclude the zero [56,57]. The S-value, i.e., the standard error of the regression, is a measure of the goodness of fit of the model under consideration instead of the R² for non-linear models [57]. The residual plots in Fig. 6(b) appear satisfactory overall, and using the Anderson–Darling test, the hypothesis of normality of the residual distribution cannot be rejected at the 95% confidence level. It should be noted that, according to the result obtained, non-linear regression is preferable to linear quadratic regression, as linearizing the function to perform linear regression can lead to bias in the predictions [58]. According to the results shown in Table 10 and Fig. 6, there is a super-linear relationship between human stress response and the complexity of product variant assembly. This result, which is one of the first attempts to study the relationship between assembly complexity and human stress response, shows that as the complexity of the product assembly increases, the assembly process becomes more challenging and entails a higher degree of mental workload and cognitive effort, leading to a more than proportional increase in human stress response.

This section introduces the “Human-Robot Collaboration Quality and Well-Being Assessment Tool” (HRC-QWAT), a tool designed to synthesize previous analyses of quality and human well-being, by directly relating H_S and D_tot, regardless of the complexity of the product assembled. The selection of total defects and human stress response in the HRC-QWAT tool was based on their significant impact on evaluating the performance of the HRC system, including both product quality and human well-being. The tool establishes a direct relationship between human stress response and total defects, enabling a comprehensive assessment of system performance within the context of single-variant and small-batch variant production. This design is specifically tailored to address the distinct challenges and demands posed by customized production scenarios, where the adaptability of the production process and the individuality of each assembly play pivotal roles.

Two typologies of HRC-QWAT are proposed. The first typology is intended for single-variant production of highly customized products produced one by one in the HRC system, even if repeated over time. This type of production involves the manufacture of a single product variant at a time, typically in response to specific customer orders or market demand. The production process is adapted as required for each variant, which can result in longer lead times and higher production costs. In this scenario, the company is interested in monitoring the performance of each individual product variant assembly in terms of quality and human well-being. On the other hand, the second typology of HRC-QWAT is proposed to provide companies with a diagnostic method for products of the same variant manufactured in small batches, after each of such productions. This type of production involves the manufacture of small batches of the same product variant, typically in response to forecasted demand or market trends. The production process is adapted for each batch, allowing a product variant to be produced more efficiently and cost-effectively than in the single variant scenario. The choice between single variant production and small-batch variant production generally depends on factors such as demand variability, lead time requirements, and production costs. Single variant production is best suited for highly customized products with low demand, while variant batch production is more efficient for producing a range of products with moderate to high demand.

The use phase of the proposed tool is the same for practitioners in both cases, while the difference lies in the realization phase of the HRC-QWAT.

In both typologies, the HRC-QWAT allows for the assessment of quality and human well-being in the collaborative assembly process, taking into account the unique characteristics and requirements of each production scenario. This tool offers a comprehensive evaluation of the HRC system's performance, considering the relationship between human stress response (H_S), total defects (D_tot), and the complexity of the assembly process. Although the complexity indicator is not explicitly included as a separate metric in the HRC-QWAT, its influence on the performance measures is implicitly accounted for in the evaluation. As discussed in the previous sections, assembly complexity plays a crucial role in affecting performance metrics, particularly in highly customized and personalized product assemblies within the same product family [59–61]. Although the HRC-QWAT does not directly measure complexity, it considers its impact on the overall performance of the collaborative process. By capturing the relationship between human stress response, total defects, and the intricate nature of the assembly, the tool indirectly accounts for the effects of assembly complexity on the HRC system's performance.

To construct the HRC-QWAT, the following operational steps should be taken. First, a set of historical experimental data representative of the production must be collected. In the case of the HRC-QWAT for single variant production, a reasonable number of products (at least about 30, for robust regression parameter estimation [62]) should be produced, and quality and human stress responses should be measured (according to Secs. 5 and 6, respectively). On the other hand, for the HRC-QWAT for small batches, an adequate number of production units should be collected for each batch (at least about fifteen units for each product type, if possible [62]) and the average performance measures should be obtained for each batch. As mentioned in Secs. 5 and 6, it is advisable to perform preliminary data analysis using conventional statistical techniques to detect and filter outliers [63].

Second, the model relating the two performance measures should be developed to represent the overall quality of the systems, in terms of product/process quality and human well-being. Considering the case study, the combination of the models in Eqs. (8) and (10) leads to a linear model. Such a linear model is the best fit when considering single variant production, as also confirmed by the goodness-of-fit statistics and residual analysis [55]. Figure 7(a) depicts the prediction model relating human stress response H_S to total defects D_tot, and Fig. 7(b) shows the residual plots. The output of the regression is shown in Table 11.

When considering small batches of products of the same variant, average values of the human stress response $(H¯S)$ and total defects $(D¯tot)$ should be obtained for each variant. Then, the prediction model should be derived using these averages. In the case study, six small batches are considered, one for each product variant (V1–V6), each consisting of six products.

Referring to the case study data, Fig. 8 illustrates the best-fitting model, i.e., a linear regression model, with the residual plots, and the main output of the regression is reported in Table 11.

The HRC-QWAT diagnostic tool (see Fig. 9) uses the model as a reference for prediction and takes into account the associated uncertainty range. Specifically, the two prediction limits (lower prediction limit, LPL; upper prediction limit, UPL), derived from the regression models shown in Figs. 7 and 8, serve as thresholds for identifying critical products and small batches, respectively. Products and small batches are classified as critical in terms of both defects and human stress response when a special source of variation, i.e., a source not inherent to the process, occurs [62]. It should be noted that the negative values of LPL are set equal to zero, as this is not physically possible. As a result, for some products or batches, the prediction interval may not be symmetrical with respect to the predicted regression value, as shown in Fig. 9.

In the use phase, when new single products or small batches of products are produced, the observed values (D_tot, H_S) or $(D¯tot,H¯S)$ are compared with the corresponding prediction limits from the HRC-QWAT for single variant or small-batch production, respectively. Accordingly:

• If the observed (D_tot, H_S) or $(D¯tot,H¯S)$ value falls within the prediction range (LPL and UPL), the product or batch is considered non-critical.

• If the observed (D_tot, H_S) or $(D¯tot,H¯S)$ value is higher than the upper prediction limit (UPL) (area A in Fig. 9) or lower than the lower prediction limit (LPL) (area B in Fig. 9), it indicates a mismatch between the human stress response and the total defects, and an abnormal situation exists, resulting in the product or batch being signaled as critical. Specifically, products or batches located in area A of Fig. 9 are reported as critical due to the high level of stress response experienced by operators compared to the number of total defects detected. On the other hand, products or batches lying in area B are characterized by abnormal defectiveness compared to the level of human stress response.

Table 12 reports an example of critical product production and an example of small-batch production detected as critical using the HRC-QWAT and possible root causes.

The proposed diagnostic tool has been developed with a dual objective. First, it aims to accurately position products or small batches on the HRC-QWAT, thereby providing a clear understanding of their relative position compared to other products. This information can be valuable in making informed quality control decisions and identifying areas for improvement. Second, the diagnostic tool is designed to detect unusual production scenarios and identify critical out-of-control situations. By continuously monitoring production processes, the tool can identify any deviations from established normal operating conditions, allowing corrective action to be taken in a timely manner. This feature of the diagnostic tool acts as an in-process control mechanism, ensuring that the quality of the overall system (product/process and human) remains consistently high throughout the production process.

In conclusion, the proposed diagnostic tool represents a significant step forward in quality control and monitoring, providing manufacturers with a powerful tool to ensure consistent product quality and to detect and correct quality deviations in real-time.

The novelty of the HRC-QWAT lies in its comprehensive assessment of quality systems, encompassing both technical aspects of production quality and the human factor of worker well-being. While previous studies have focused on individual measurements and indicators in HRC systems, the HRC-QWAT combines multiple dimensions of evaluation to provide a more holistic understanding of the collaborative assembly process. By integrating indicators related to total defects and human stress response, the tool offers a more nuanced evaluation of the performance of HRC systems.

Moreover, the HRC-QWAT's versatility and adaptability contribute to its novelty. It can be applied to both single variant and small-batch production scenarios, accommodating different production environments and collaboration settings. Whether the work is predominantly performed by a robot or in a high-intensity human work environment, the HRC-QWAT assesses production quality and worker stress response, ensuring an optimized collaborative process. The tool's adaptability allows it to be fine-tuned to the unique parameters of various production environments and collaboration scenarios, making it not only a quality and well-being assessment tool but also a strategic tool for comparing and contrasting different collaboration scenarios.

The potential for generalization is another key aspect of the HRC-QWAT's novelty. Although the case study focused on electronic board assembly, the design and methodology of the HRC-QWAT were conceived with a broader application in mind. Its adaptability allows it to be utilized in a wide range of production scenarios, even when a robot performs the majority of the work and the role of the human operator is minimal or focused on labor-intensive tasks. It should be noted, however, that the generalizability of the HRC-QWAT depends on careful adaptation and refinement of the model parameters. This will allow the tool to accurately reflect the interaction dynamics and associated stress responses in different HRC settings. The possibility of extending the use of the HRC-QWAT to more diverse and nuanced collaboration scenarios represents a promising avenue for future research and development in the field of HRC.

The aim of the present research was to propose a novel tool, called the HRC-QWAT, which combines two indicators to evaluate and monitor the quality of a production system: the total number of defects generated during the production of product variants, and the stress response of workers. This innovative tool addresses a significant gap in the field of human–robot collaboration assessment, providing a unique approach to evaluating both the production quality and the well-being of human operators. The methodology used a collaborative human–robot assembly system as a case study to demonstrate the feasibility of the HRC-QWAT approach. The methodology consists of two main phases: (1) the realization phase, in which the HRC-QWAT is constructed by collecting historical experimental data and developing a model relating the two performance measures (total defects and human stress response) that represent the overall quality of the system; and (2) the use phase, in which the HRC-QWAT is used as a reference for predicting future products/batches and identifying critical products in terms of defects and human stress response. The diagnostic tool uses the model to compare observed performance measures with corresponding prediction limits and detect abnormal production scenarios.

The HRC-QWAT introduces a novel approach to the evaluation of quality systems in HRC. Unlike previous studies that focused on individual metrics, this tool comprehensively assesses both technical production quality and worker well-being factors. Its adaptability and versatility make it suitable for single variant or small-batch production and for different environments and collaborative settings. In addition, although in this study the HRC-QWAT was applied to electronic boards assembly, its adaptable design allows for a broader application, opening doors for future research in the evaluation and development of human–robot collaboration.

A limitation of the proposed approach is the use of a structural complexity model, which was originally designed for manual and fully automated processes. While such a model serves as a good first approximation for collaborative human–robot contexts, as the cobot mainly performs logistical and organizational support tasks, a more refined complexity model will be required for a more accurate evaluation. Another limitation of the study is that the comparison in the HRC-QWAT is based on only two indicators, total defects and human stress response. In such measures, the performance of the cobot is not directly evaluated, although it is implicitly reflected in the total number of defects. Furthermore, additional performance measures, such as workload, are not directly addressed. Recognizing these limitations, it is important to consider that the proposed approach has the flexibility to be extended to include additional indicators, including cobot’s performance and workload, as well as encompass process sustainability and economic impact measures.

Future research efforts will aim to overcome (at least some of) the above-mentioned limitations. Particular attention will be paid to refining the complexity model by including factors related to HRC and performing a validation of the proposed approach using different products to quantitatively assess its efficiency. In addition, the study could be extended to include other cobot performance measures, including efficiency metrics (cycle time, throughput), accuracy and reliability metrics, safety metrics, and environmental/economic sustainability indicators, such as equivalent carbon dioxide emissions and life cycle costs.

There are no conflicts of interest.

The data sets generated and supporting the findings of this article are obtainable from the corresponding author upon reasonable request.

C =

assembly complexity

S =

standard error of the regression

N =

total number of product components

R² =

coefficient of determination

a_min =

minimum amplitude of the SRC peaks

a_max =

maximum amplitude of the SRC peaks

a_w =

amplitude of the w-th SCR peak

c_max =

theoretical maximum value for the connection index

c_pr =

complexity in achieving a connection between components p and r

e_pr =

${1,ifthereisaconnectionbetweenpandr0,otherwise$

e_pr =

(p,r)th entry of the AM matrix of the product

h_max =

theoretical maximum value for the handling index

h_p =

handling complexity of component p

k₁ =

regression coefficient of the model D_tot versus C

k₂ =

regression coefficient of the model H_S versus C

k₄ =

regression coefficient of the model H_S versus D_tot

k₃ =

regression coefficient of the model H_S versus C

$t1−α2,γ$ =

value of the Student's t distribution with γ degrees-of-freedom and significance level α

C₁ =

handling complexity

C₂ =

complexity of connections and liaisons

C₃ =

topological complexity

E_AM =

matrix energy of AM

D₁ =

in-process defects

D₂ =

offline defects

D_tot =

total number of defects

H_S =

human stress response

N_B =

number of applicable handling difficulties related to attribute B

N_P =

total number of SCR peaks

$dhi∈{A,B,C,D}$ =

handling difficulty of attribute i

$dcj∈{E,F,G,H,I,J,K}$ =

connection difficulty of attribute j

$D¯tot$ =

average value of total defects

$H¯S$ =

average value of human stress response

$H⌢S$ =

predicted H_S value of the regression curve

AM =

binary adjacency matrix of the product

CI =

confidence interval

EDA =

electrodermal activity

HRC =

human–robot collaboration

HRC-QWAT =

human–robot collaboration quality and well-being assessment tool

SCR =

skin conductance response

SE(Fit) =

standard error of the fit

SE =

standard error

PI =

prediction interval

LPL =

lower prediction limit

UPL =

upper prediction limit

δ_q =

singular values of AM