Abstract

Improper controller parameter settings in physical human–robot interaction (pHRI) can lead to instability, compromising both safety and system performance. This study investigates the relationship between cognitive and physical aspects of co-manipulation by leveraging electroencephalography (EEG) to predict instability in physical human–robot interaction. Using elastic net regression and deep convolutional neural networks, we estimate instability as subjects guide a robot through predefined trajectories under varying admittance control settings. Our results show that EEG signals can predict instability up to 2 s before it manifests in force data. Moreover, the deep learning-based approach significantly outperforms elastic regression, achieving a notable (∼10%) improvement in predicting the instability index. These findings highlight the potential of EEG-based monitoring for enhancing real-time stability assessment in pHRI.

1 Introduction

Physical human–robot interaction (pHRI) aims to integrate robotic systems with human cognitive abilities to perform tasks that exceed the capabilities of either entity alone [1]. Two critical aspects must be considered: (1) ensuring safety and reliability through the proper design and control of robotic systems [2,3], and (2) modeling human cognition to predict intentions [4] and responses to robot actions [5,6]. These aspects are deeply interconnected, as physical interactions can be modulated to elicit specific cognitive responses, while cognitive states can reflect the preferred interaction parameters [4,7,8] or modulate feedback from robots to humans [9,10] to enhance perceived safety [11]. For example, in rehabilitation, haptic controllers provide feedback to patients, with their cognitive engagement guiding the adaptation of controller parameters [8,12]. The need for assessing the quality of human–robot interaction using human cognition is even more critical in physical interaction.

When robots physically interact with human operators, it is impossible to model all the human actions to ensure safety and dependability. As a result, compliant control strategies such as admittance and impedance control [13] are utilized to accommodate human operators' natural and unstructured movements [14,15]. These strategies empower a rigid actuator, equipped with either a position or force sensor, to manifest compliant behavior by simulating virtual dynamics via manipulating virtual inertia, damping, or stiffness control parameters. While compliance is crucial to ensuring interaction safety, it alone does not suffice for achieving an intuitive pHRI. Alongside compliance, adaptability to diverse tasks becomes imperative. Studies indicate that controllers adjusting their parameters in accordance with task requirements and human intent outperform constant impedance/admittance controllers. Nonetheless, incorrect control parameter settings can precipitate unstable interactions, thereby compromising operator safety and the system functionality [16].

To minimize effort and maintain stability, the interaction should be continuously assessed, and control parameters should be adjusted in real-time [3,17,18]. Several studies have focused on real-time stability prediction using position [19] or velocity profile analysis [20,21]. Alternative approaches based on frequency-domain analysis have extended stability assessment to interaction forces. For example, Balasubramanian et al. [22] introduced the concept of “spectral arc length,” which evaluates interaction smoothness by analyzing Fourier spectrum variations in speed profiles. While this metric accommodates complex movements, its robustness in co-manipulation scenarios is significantly influenced by individual arm stiffness and task-specific characteristics.

A more reliable approach was proposed by Ryu et al. [23] and later adapted for admittance control by Dimeas and Aspragathos [24]. This method distinguishes between low-frequency components of intended human motion and high-frequency oscillations caused by instability, forming an instability index. We leverage this frequency-based interaction instability index (IR) to account for individual variations and user intent.

Instability undermines the efficacy of human-coupled robotic systems and poses risks to user well-being [17,25]. Therefore, investigating the relationship between the cognitive and physical facets of pHRI is crucial for achieving a safer and more organic interaction. The human reaction time to tactile stimulation associated with instability typically ranges from 150 ms to 400 ms [26]. When additional cognitive processes such as motion planning and visual perception are involved (e.g., steering a vehicle), this response time can extend up to 700 ms [27]. Hence, the fast detection time (or even prediction) of any instability in human–robot interaction can help the real-time adjustment of robot behavior to ensure safer physical interaction. In this regard, there have been recent studies that indicate a potential correlation between electroencephalography (EEG) activities and haptic feedbacks in physical human–robot interaction [2830]. Drawing inspiration from EEG studies in driving assistance systems [3133], where braking intention has been detected from brain activity up to 380 ms before foot engagement with the brake pedal [34]. We hypothesize that instability in human–robot interaction can be detected from EEG data before they manifest in physical interaction (force data). This proactive detection can potentially enable earlier intervention, reducing reliance on delayed physical responses.

This article investigates the effectiveness of brain activity monitoring in predicting the instability index during physical human–robot interaction and examines how early EEG signals can anticipate instability before they become observable in force data. To achieve this, we conducted a pHRI experiment (Fig. 1) in which participants guided a robot under admittance control along a predefined trajectory. By adjusting the admittance control parameters—some of which were intentionally suboptimal for the task—we induced varying levels of interaction stability. Interaction force data were recorded and analyzed to compute the true instability index. Simultaneously, EEG data were collected and used to predict the instability index via a deep convolutional neural network (CNN) and an elastic net regression (ENR) approaches.

Fig. 1
Overview of this study. Motor control tasks are performed in a pHRI experiment with various admittance control settings. Force data are used to calculate the instability index in the interaction. Brain activity is used in a deep learning framework to estimate those indexes.
Fig. 1
Overview of this study. Motor control tasks are performed in a pHRI experiment with various admittance control settings. Force data are used to calculate the instability index in the interaction. Brain activity is used in a deep learning framework to estimate those indexes.
Close modal

Furthermore, to assess the temporal advantage of EEG-based predictions, we incrementally shifted the prediction window by 0.5 s up to 2 s before the instability was reflected in force data. This allowed us to evaluate how early brain activity can signal impending instability compared to interaction dynamics.

2 Materials and Methods

2.1 Experiment Design.

In an experiment approved by the Institutional Review Board, 11 subjects (8 males and 3 females; all right-handed) were recruited from the University of Buffalo School of Engineering, Buffalo, NY. Participants' age ranged from 23 to 34 years (mean = 24, SD = 2.1). The participants were asked to hold the end-effector of a robot with a force–torque sensor and guide it in between a star-shaped trajectory as shown in Fig. 2.

Fig. 2
Tracked path and boundary in (a) low- and (b) high-damping conditions
Fig. 2
Tracked path and boundary in (a) low- and (b) high-damping conditions
Close modal

The robot operates under an admittance control framework in which human intent is detected using a force–torque sensor. The robot's end-effector position or velocity is determined based on the magnitude and the direction of the force exerted by the human operator. The robot's response is governed by a virtual dynamic model, which includes parameters such as mass, damping, and stiffness. By adjusting these parameters, the system's sensitivity, stability, and the level of human effort required for interaction can be optimized [16,28,35].

The design of the admittance controller is depicted in Fig. 3. The velocity feedback sets up a first-order relationship between the input force (Fh) and the end-effector velocity (vc) in Cartesian space given by Eq. (1)
(1)
Fig. 3
Experiment and admittance controller design
Fig. 3
Experiment and admittance controller design
Close modal

In this equation, Md and Cd represent the virtual mass (inertia) and virtual damping coefficient matrices (set as diagonal matrices), respectively, which are defined for each joint of the robotic system. Here, the diagonal elements are set as Md = [3, 3, 3, 0.1, 0.1, 0.1]. Cd is used to adjust the robot's response by transitioning between high and low damping where an increase enhances motion stability but reduces human comfort, and vice versa. Refer to Ref. [17] for more details on the effect of virtual damping on human effort and motion stability.

Integrating Eq. (1) yields the end-effector velocity, allowing joint velocities (q˙) to be derived from the desired end-effector velocity and the Jacobian matrix (J) in Eq. (2).
(2)

The experiment involved altering the controller dynamics of the robot by adjusting the damping coefficient (Cd) in the admittance control while keeping other parameters constant. Participants completed separate trials under low-damping (high admittance) and high-damping (low admittance) settings. The high-damping parameters are set as Cd = Diag([80, 80, 80, 10, 10, 10]) and the low-damping parameters are set as Cd = Diag([20, 20, 20, 6, 6, 6]). Low-damping conditions reduced the resistance from the robot, consequently demanding less effort from the human operator to maneuver the end effector. Conversely, high damping increased resistance, elevating the effort required for manipulation. However, under low damping, the robot exhibited heightened sensitivity to force variations, potentially leading to instability during co-manipulation.

2.2 Data Collection and Preprocessing.

To assess the stability of human–robot interactions, we utilized both EEG data and interaction forces between the participant and the robot. The EEG signals were collected using a B-Alert X20 wireless headset (Advanced Brain Monitoring, Carlsberg, CA) at 20 locations (Fp1, Fp2, F7, F3, Fz, F4, F8, T3, C3, Cz, C4, T4, T5, P3, Pz, P4, T6, O1, POz, and O2) and referenced with regard to electrodes located behind each ear on the mastoid bone. The interaction forces were measured using a six-axis force–torque sensor at the robot's end effector.

EEG signals were preprocessed to remove electrical noises by applying band-pass filtering to the EEG signals within the frequency range of 0.3–50 Hz. Subsequently, the EEG signals are epoched into 2-second segments and time-synchronized with force data; any identified bad epochs are excluded from further analysis. We employ independent component analysis with the Picard algorithm to eliminate eye blinks and muscle artifacts utilizing the mne-python library [36].

2.3 Instability Index Quantification.

To quantify the instability of the human–robot interaction, we propose a novel, dimensionless instability index to quantify both low-frequency intended human arm movements and high-frequency unstable oscillations in robots. The premise relies on the distinction in frequency bandwidths between human–arm movements, which typically operate within a relatively low-frequency range [37], and the unstable oscillations in robots resulting from inadequately set admittance levels.

Researchers have extensively explored frequency-domain analysis of end-effector position and interaction force to identify and mitigate unstable oscillations in haptic devices [23,24]. However, the suitability of this analysis depends on the specific type of robot controller being utilized. For instance, Dimeas and Aspragathos [24] demonstrated that spectral analysis of position data remains largely effective for back-drivable haptic devices. In contrast, spectral analysis of interaction forces proves to be more pertinent for robots controlled by admittance. This preference arises because an admittance controller functions like a low-pass filter, attenuating high-frequency force components.

Inspired by Refs. [23,24], our study introduces a dimensionless index to quantify instability within a physical human–robot interaction experiment. This index utilizes frequency-domain analysis of interaction force as Eq. (3).
(3)

Here Pf(ω) is the fast Fourier transform (FFT) of interaction forces at a frequency of ω, ω0 is the smallest non-zero frequency component of the FFT, ωm is the maximum frequency component of interest for summation, which must be less than Nyquist frequency, ωc is the cross-over frequency to distinguish between intended and unintended motions which is between ω0 and ωm. For FFT, the force data were down-sampled to 128 Hz and then segmented to a 2-second data epoch (aligned with EEG signals). A Hamming window with 50% overlap was applied to each segment. Periodograms were averaged to estimate spectral power, and results were normalized according to the overall window power.

The instability index (IR) ranges from 0 to 1, with higher values representing higher instability and lower values representing lower interaction instability. The proposed instability index (IR) has two parameters namely ωm and ωc. The ωm should be chosen so that the effect of sensor noise is minimal and around the maximum range of robot motion. However, the ωc value is intricate, as it is influenced by individual factors such as the user's arm impedance, skill level, and task type. Consequently, diverse users may exhibit distinct IR values for a given task due to variations in ωc. To compensate for this individual difference, in this work, the mean value of IR indices is mapped to zero for each subject.

2.4 Regression Analysis.

The overall pipeline for predicting the instability index from EEG signals is shown in Fig. 4. We use two different regression approaches: one based on feature extraction (ENR) and the other on raw data (CNN). Each method has strengths and assumptions, influencing how the EEG data are processed. In the first approach, we use ENR, which involves extracting 120 spectral power features from 2-second EEG epochs. We apply a logarithmic transformation to normalize the spectral data, as suggested by Gasser et al. [38]. We also standardize the spectral power values by converting them to z-scores based on each subject's distribution. This approach focuses on deriving meaningful, hand-crafted features from the EEG signals for regression.

Fig. 4
Regression pipeline using the EEG data
Fig. 4
Regression pipeline using the EEG data
Close modal

In contrast, the second approach uses a CNN that directly processes raw EEG epochs without any feature extraction. The CNN automatically learns relevant patterns from the raw data, avoiding the need for manual feature engineering.

We use leave-one-subject-out cross-validation to evaluate the performance of both approaches. In each fold, the model is trained on all subjects except the tested one (see Fig. 4). For elastic net, a tenfold cross-validation tunes parameters via grid search, while for CNN, a fivefold cross-validation is used due to higher computational demands of deep learning models.

2.4.1 Elastic Net Regression.

To predict the instability index, the first model we employed is elastic net regression, a regularized linear regression method that combines the strengths of lasso (L1) and ridge (L2) regression. The elastic net regression is particularly useful when dealing with high-dimensional data, as it performs both coefficient shrinkage and feature selection. The objective function for elastic net is given by Eq. (4) where the first term is the least square function, and the second term corresponds to the regularization function. Pα is described in Eq. (5)
(4)
(5)

In Eq. (4), the scalar weight λ and the parameter α in Eq. (5) (ranging from 0 to 1) balance the penalty terms, offering a continuum between the L1 norm and squared L2 norm of the vector w [39]. When α tends toward 0, the elastic net method resembles ridge regression, whereas an α value of 1 reflects lasso regression. Solving this regularized optimization problem demands numerical optimization techniques. Ridge regression induces coefficient shrinkage toward zero but does not eliminate any coefficients, retaining all features in the model. Conversely, lasso regression achieves both coefficient shrinkage and feature selection simultaneously. Consequently, some features are assigned zero weights, effectively removing them from the model. The model was trained to predict instability based on the 120 spectral power features derived from the EEG signals.

2.4.2 Convolution Neural Network.

Alternatively, we also utilized a deep learning approach with a CNN to predict the instability index. CNNs have shown great success in processing time-series data like EEG signals due to their ability to learn spatial and temporal features automatically. The CNN architecture illustrated in Fig. 5 draws inspiration from Refs. [40] and [41].

Fig. 5
CNN architecture used in this study
Fig. 5
CNN architecture used in this study
Close modal
The input EEG data consist of a 2-second epoch with dimensions n × d, where n is the number of electrodes (20 in our case) and d is the number of samples per epoch (256). It is first transformed using two linear operations (Eq. 6). The first transformation normalizes the data by subtracting the mean of each electrode (x¯) and scaling it by the standard deviation (x~sh) which is followed by a shift and scale operation using learnable parameters Wsh and Wsc.
(6)

Next, the data are passed through convolution layers. The first convolution (temporal) uses a 1 × 10 kernel with a stride of 1, producing three channels (using three filters). The second convolution applies a 20 × 20 kernel to capture spatial correlations across electrodes, also with a stride of 1. The output is then squared element-wise, followed by batch normalization and average pooling with a 1 × 50 kernel and a stride of 1 × 10. The pooled output undergoes a logarithmic transformation before passing through another convolution and two fully connected layers with a tanh activation function. The overall architecture is illustrated in Fig. 6. We trained the CNN using the adam optimizer, with an average squared error and a learning rate of 1×103. Training is done for 150 epochs using Nvidia Titan Xp GPU and the PyTorch library.

Fig. 6
Trainable weights of the shift-scale network
Fig. 6
Trainable weights of the shift-scale network
Close modal

2.5 Instability Index Prediction Setup.

The instability index IR was predicted using EEG data, and we tested different overlap conditions between EEG epochs to assess whether we could predict the instability before it happens. Specifically, we explored temporal alignments ranging from 0% to 100% overlap between EEG epochs and instability occurrences (Fig. 7).

Fig. 7
Different overlap conditions to predict the IR index using EEG with (a) 0%, (b) 50%, and (c) 100% overlaps
Fig. 7
Different overlap conditions to predict the IR index using EEG with (a) 0%, (b) 50%, and (c) 100% overlaps
Close modal

We tested five overlap conditions, increasing from 0% to 100% in 25% increments. A 0% overlap corresponds to an early prediction of instability by 2 s relative to force data. As the overlap increases (e.g., 25%, 50%, 75%), the prediction becomes less early, gradually approaching real-time detection. At 100% overlap, the focus shifts entirely to correlation analysis rather than early prediction.

3 Results and Discussion

The result of frequency analysis of the interaction forces and the individualized instability index (IR) are presented in this section followed by the outcomes of the regression analysis for predicting IR from EEG.

The average power spectral density (PSD) of interaction force across subjects is shown in Fig. 8. The power of interaction force is significantly higher in the high-damping compared to the low-damping condition, indicating higher interaction force (human effort). This distinction is visually evident in the vertical shift of the PSD in Fig. 8(a). However, this difference may introduce a potential bias in frequency analysis and instability index calculation. To address this issue, we propose scaling the PSD by the DC gain (Pf(ω=0)), as illustrated in Fig. 8(b). This adjustment is essential for mitigating the impact of varying damping conditions on the observed PSD and instability index calculation. Figure 8(b) further reveals that the low-damping condition is associated with higher frequency components beyond human voluntary motion, which corresponds to instability in the interaction force.

Fig. 8
(a) Raw and (b) scaled PSD of interaction forces averaged over all subjects. The shaded error area is one standard deviation.
Fig. 8
(a) Raw and (b) scaled PSD of interaction forces averaged over all subjects. The shaded error area is one standard deviation.
Close modal

To determine the instability index (IR), the maximum frequency (ωm) is set at 10 Hz, based on the averaged power spectral results. However, individualizing the cross-over frequency (ωc) is necessary to account for inter-subject variations in arm impedance and motor control characteristics. This customization is achieved through statistical analysis under both high and low damping conditions. For each subject, the instability index is computed by varying ωc from 0.5 Hz to 4 Hz in 0.25 Hz increments. At each step, the divergence in IR between high and low damping is quantified using F-statistics from analysis of variance (ANOVA). This analysis reveals the trend of F-statistics concerning ωc, with the optimal ωc identified at the point where the difference in IR between high and low damping conditions is maximized. This process is shown in Fig. 9. Optimal ωc values, exemplified in Fig. 10 for subjects 5 and 6, are obtained where the F-statistics peak. The optimal cross-over frequencies (ωc) of all subjects are listed in Table 1.

Fig. 9
The F-statistic values obtained from ANOVA on IR distributions for different ωc. The cross-over frequency is selected to maximize the F-statistics.
Fig. 9
The F-statistic values obtained from ANOVA on IR distributions for different ωc. The cross-over frequency is selected to maximize the F-statistics.
Close modal
Fig. 10
The selected cross-over frequency ωc for each subject based on the maximum F-statistics
Fig. 10
The selected cross-over frequency ωc for each subject based on the maximum F-statistics
Close modal
Table 1

Individualized cross-frequency ωc in Hertz

SubS1S2S3S4S5S6S7S8S8S10S11Ave
ωc2.751.52.03.02.53.52.02.03.03.01.52.43
SubS1S2S3S4S5S6S7S8S8S10S11Ave
ωc2.751.52.03.02.53.52.02.03.03.01.52.43

This approach requires a significant difference between high- and low-damping conditions to estimate ωc. To verify that, we performed a paired t-test on the averaged (across subjects) IR index, which revealed a strong statistical difference (t(11) = −15.1, p < 107) indicating that IR was indeed higher in the low-damping than high-damping condition, validating this index as a quantification of motor control difficulty.

While individualized ωc enhances IR index estimation, it concurrently amplifies between-subject differences. Subjects with lower ωc exhibit larger IR indices, whereas those with higher ωc demonstrate smaller IR indices. Thus, individualized ωc is more appropriate for within-subject analysis rather than between-subject comparisons. To mitigate this issue, an averaged ωc (Table 1) is employed for subsequent analysis. Notably, this average ωc value (2.35 Hz) aligns closely with findings from studies on tremor identification [42,43].

For each subject, the IR index is estimated from EEG data across approximately 600 epochs using CNN and ENR. The distribution of the predicted index for four subjects is shown in Fig. 11.

Fig. 11
Comparison of the predicted and actual IR for low- and high-damping conditions
Fig. 11
Comparison of the predicted and actual IR for low- and high-damping conditions
Close modal

Despite the differences between actual and estimated IR indices, the CNN model demonstrates reliable performance, particularly in distinguishing between low- and high-damping conditions (p-value < 105). The root mean square error (RMSE) of the estimated IR is averaged for each subject (S1–S11) and compared across different overlap percentages. Table 2 presents this comparison for both approaches. Notably, CNN exhibits superior performance on average compared to elastic net regression. To assess the statistical significance of this improvement, a Wilcoxon signed-rank test is conducted, with results detailed in Table 3.

Table 2

Root mean square error of estimated IR indices using ENR and CNN models

ModelOverlapS1S2S3S4S5S6S7S8S9S10S11
ENR0%0.05060.06470.05310.06950.04330.07150.05530.04530.04190.04660.0551
25%0.04850.06640.05230.06620.04320.07260.05790.04580.04080.04660.0583
50%0.05240.06450.05370.07110.04660.07090.05640.04620.04040.04520.0544
75%0.04530.06350.04980.06750.04080.06940.05340.04550.03980.04720.0559
100%0.04370.06320.04970.06440.04170.06910.05680.04410.03850.04710.0546
CNN0%0.04020.06120.04990.06130.03950.06550.05780.04130.03580.03520.06
25%0.04520.05680.05750.05970.04010.06680.05650.0430.03630.03790.06
50%0.04450.06280.05050.06060.04530.06620.06340.03980.03560.0340.0603
75%0.04640.06350.05690.05910.03550.06640.05950.04120.03510.03480.0594
100%0.0460.06320.05580.05850.04180.06630.05890.04170.0330.03390.0595
ModelOverlapS1S2S3S4S5S6S7S8S9S10S11
ENR0%0.05060.06470.05310.06950.04330.07150.05530.04530.04190.04660.0551
25%0.04850.06640.05230.06620.04320.07260.05790.04580.04080.04660.0583
50%0.05240.06450.05370.07110.04660.07090.05640.04620.04040.04520.0544
75%0.04530.06350.04980.06750.04080.06940.05340.04550.03980.04720.0559
100%0.04370.06320.04970.06440.04170.06910.05680.04410.03850.04710.0546
CNN0%0.04020.06120.04990.06130.03950.06550.05780.04130.03580.03520.06
25%0.04520.05680.05750.05970.04010.06680.05650.0430.03630.03790.06
50%0.04450.06280.05050.06060.04530.06620.06340.03980.03560.0340.0603
75%0.04640.06350.05690.05910.03550.06640.05950.04120.03510.03480.0594
100%0.0460.06320.05580.05850.04180.06630.05890.04170.0330.03390.0595
Table 3

Comparison of RMSE of CNN and ENR models in estimating IR index averaged over all subjects

OverlapCNNENRImprovementP-value
0%0.04970.055510.45%0.018a
25%0.05090.05578.61%0.032a
50%0.05120.05628.89%0.101
75%0.05070.05385.76%0.386
100%0.05080.05344.86%0.507
OverlapCNNENRImprovementP-value
0%0.04970.055510.45%0.018a
25%0.05090.05578.61%0.032a
50%0.05120.05628.89%0.101
75%0.05070.05385.76%0.386
100%0.05080.05344.86%0.507
a

Indicates significant difference.

The results in Table 3 indicate that CNN has notably improved the prediction of IR by up to ∼10.45% compared to ENR. However, this improvement is statistically significant only at 0% and 25% overlap conditions (p-value < 0.05) and not at higher overlap levels. To further assess the magnitude of this improvement, we perform Cohen's d test, which measures the effect size—a standardized measure of the difference between two means. Effect size quantifies the practical significance of the difference, independent of sample size.

In this analysis, 0% overlap has the largest effect size (d = 0.406), indicating a moderate difference between ENR and CNN. The 25% and 50% overlaps show smaller yet noticeable effect sizes (d = 0.33 and d = 0.32, respectively), while 75% and 100% overlap have the smallest effect sizes (d = 0.16 and d = 0.12), suggesting minimal differences between the two models at higher overlap levels. This analysis confirms that the largest differences between the two models occur at lower overlap conditions (0% and 25%), aligning with the Wilcoxon test results, which found significant differences at these levels. This suggests that CNN is a more reliable model for early prediction of instability. However, for real-time detection (higher overlap between force and EEG data), there is no significant difference between the two models.

4 Conclusion

The stability of co-manipulation depends on the proper adjustment of the virtual damping coefficient in an admittance control strategy. In this study, we explored the feasibility of predicting the instability index from brain activity to enhance adaptive control. As a reference measure, we employed frequency-domain analysis of force data using a 2-second moving window. To predict the instability index, we applied: (1) ENR on power spectral density and connectivity features of EEG data and (2) a deep CNN trained on artifact-free EEG data. Both models demonstrated a reliable estimation performance; however, the estimation quality varied between subjects. The main factor contributing to the varied estimation quality is the functional differences in the subject's motor control skills, which influences their brain activity. This issue can be potentially overcome by training an individualized regression model.

It should also be noted, while the IR index serves as a measure of instability, the precise prediction of its numerical value is less critical in practice than the ability to reliably distinguish between stable and unstable states. Such classification would significantly aid the real-time adjustment of admittance control parameters. Notably, the proposed CNN-based approach successfully and significantly distinguished between stable and unstable conditions, demonstrating its potential for enhancing adaptive human–robot interaction. In future work, we plan to develop an individualized classifier to detect unstable modes from the predicted IR, enabling real-time admittance control adjustments. A promising direction for improving robustness is leveraging an ensemble approach that combines ENR and CNN predictions, integrating their strengths to enhance stability detection and control adaptability.

Furthermore, to evaluate the temporal advantage of EEG-based predictions, we shifted the prediction window up to 2 s before instability was reflected in the force data. Notably, the CNN model maintained its performance even with the maximum shift, whereas the elastic net regression model showed a significant decline in performance beyond a 250 ms shift. This finding highlights the potential of EEG-based instability prediction as a proactive measure in physical human–robot interaction. However, it is important to acknowledge the limitations of this study, which involved only 11 subjects performing fine manipulation tasks within a single session. Further research is needed to assess the impact of within-session variability and validate these findings in a larger, more diverse population.

Conflict of Interest

There are no conflicts of interest.

Data Availability Statement

The datasets generated and supporting the findings of this article are obtainable from the corresponding author upon reasonable request.

References

1.
Goodrich
,
M. A.
, and
Schultz
,
A. C.
,
2008
, “
Human–Robot Interaction: A Survey
,”
Found. Trends Hum. Comput. Interact.
,
1
(
3
), pp.
203
275
.
2.
De Santis
,
A.
,
Siciliano
,
B.
,
De Luca
,
A.
, and
Bicchi
,
A.
,
2008
, “
An Atlas of Physical Human–Robot Interaction
,”
Mech. Mach. Theory
,
43
(
3
), pp.
253
270
.
3.
Villani
,
V.
,
Pini
,
F.
,
Leali
,
F.
, and
Secchi
,
C.
,
2018
, “
Survey on Human–Robot Collaboration in Industrial Settings: Safety, Intuitive Interfaces and Applications
,”
Mechatronics
,
55
, pp.
248
266
.
4.
Sirintuna
,
D.
,
Ozdamar
,
I.
,
Aydin
,
Y.
, and
Basdogan
,
C.
,
2020
, “
Detecting Human Motion Intention During pHRI Using Artificial Neural Networks Trained by EMG Signals
,”
2020 29th IEEE International Conference on Robot and Human Interactive Communication (RO-MAN)
,
Naples, Italy
,
Aug. 31–Sept. 4
, IEEE, pp.
1280
1287
.
5.
Xing
,
B.
, and
Marwala
,
T.
,
2018
, “Introduction to Human Robot Interaction,”
Smart Maintenance for Human–Robot Interaction
,
B.
Xing
, and
T.
Marwala
, eds.,
Springer
,
New York
, pp.
3
19
.
6.
Mutlu
,
B.
,
Roy
,
N.
, and
Sabanovic
,
S.
,
2016
, “Cognitive Human–Robot Interaction,”
Springer Handbook of Robotics
,
B.
Siciliano
, and
O.
Khatib
, eds.,
Springer
,
New York
, pp.
1907
1934
.
7.
Memar
,
A.
, and
Esfahani
,
E.
,
2018
, “
Using EEG for Predicting User Preferences of Physical Compliance in Human-Robot Cooperation
,”
2018 2nd International Neuroergonomics Conference
,
Philadelphia, PA
,
June 27–29
, pp.
1
2
.
8.
Mashayekhi
,
M.
, and
Moghaddam
,
M. M.
,
2022
, “
EMG-Driven Fatigue-Based Self-Adapting Admittance Control of a Hand Rehabilitation Robot
,”
J. Biomech.
,
138
, p.
111104
.
9.
Distefano
,
J. P.
,
Chowdhury
,
S.
, and
Esfahani
,
E.
,
2024
, “
Exploring Human-Swarm Interaction Dynamics in Cyber-Physical Systems: A Physiological Approach
,”
J. Integr. Des. Process Sci.
, p.
10920617241292155
.
10.
Li
,
S.
,
Zheng
,
P.
,
Liu
,
S.
,
Wang
,
Z.
,
Wang
,
X. V.
,
Zheng
,
L.
, and
Wang
,
L.
,
2023
, “
Proactive Human–Robot Collaboration: Mutual-Cognitive, Predictable, and Self-Organising Perspectives
,”
Rob. Comput.-Integr. Manuf.
,
81
, p.
102510
.
11.
Rubagotti
,
M.
,
Tusseyeva
,
I.
,
Baltabayeva
,
S.
,
Summers
,
D.
, and
Sandygulova
,
A.
,
2022
, “
Perceived Safety in Physical Human–Robot Interaction-A Survey
,”
Rob. Auton. Syst.
,
151
, p.
104047
.
12.
Manjunatha
,
H.
,
Pareek
,
S.
,
Memar
,
A. H.
,
Kesavadas
,
T.
, and
Esfahani
,
E. T.
,
2020
, “
Effect of Haptic Assistance Strategy on Mental Engagement in Fine Motor Tasks
,”
J. Med. Rob. Res.
,
5
(
01n02
), p.
2041004
.
13.
Calanca
,
A.
,
Muradore
,
R.
, and
Fiorini
,
P.
,
2015
, “
A Review of Algorithms for Compliant Control of Stiff and Fixed-Compliance Robots
,”
IEEE/ASME Trans. Mechatron.
,
21
(
2
), pp.
613
624
.
14.
Djuric
,
A. M.
,
Urbanic
,
R.
, and
Rickli
,
J.
,
2016
, “
A Framework for Collaborative Robot (Cobot) Integration in Advanced Manufacturing Systems
,”
SAE Int. J. Mater. Manuf.
,
9
(
2
), pp.
457
464
.
15.
Hu
,
M.
,
2023
, “
Research on Safety Design and Optimization of Collaborative Robots
,”
Int. J. Intell. Rob. Appl.
,
7
(
4
), pp.
795
809
.
16.
Keemink
,
A. Q.
,
van der Kooij
,
H.
, and
Stienen
,
A. H.
,
2018
, “
Admittance Control for Physical Human–Robot Interaction
,”
Int. J. Rob. Res.
,
37
(
11
), pp.
1421
1444
.
17.
Memar
,
A. H.
, and
Esfahani
,
E. T.
,
2019
, “
Objective Assessment of Human Workload in Physical Human-Robot Cooperation Using Brain Monitoring
,”
ACM Trans. Hum.-Rob Interact.
,
9
(
2
), pp.
1
21
.
18.
Hemanth
,
M.
,
Jujjavarapu
,
S. S.
, and
Esfahani
,
E. T.
,
2020
, “
Classification of Motor Control Difficulty Using EMG in Physical Human-Robot Interaction
,”
2020 IEEE International Conference on Systems, Man, and Cybernetics (SMC)
,
Toronto, ON, Canada
,
Oct. 7–10
, IEEE, pp.
2708
2713
.
19.
Balasubramanian
,
S.
,
Melendez-Calderon
,
A.
,
Roby-Brami
,
A.
, and
Burdet
,
E.
,
2015
, “
On the Analysis of Movement Smoothness
,”
J. Neuroeng. Rehabil.
,
12
(
1
), p.
112
.
20.
Teo
,
C. L.
,
Burdet
,
E.
, and
Lim
,
H.
,
2002
, “
A Robotic Teacher of Chinese Handwriting
,”
Proceedings 10th Symposium on Haptic Interfaces for Virtual Environment and Teleoperator Systems (HAPTICS 2002)
,
Orlando, FL
,
Mar. 24–25
, IEEE, pp.
335
341
.
21.
Vaisman
,
L.
,
Dipietro
,
L.
, and
Krebs
,
H. I.
,
2012
, “
A Comparative Analysis of Speed Profile Models for Wrist Pointing Movements
,”
IEEE Trans. Neural Syst. Rehabil. Eng.
,
21
(
5
), pp.
756
766
.
22.
Balasubramanian
,
S.
,
Melendez-Calderon
,
A.
, and
Burdet
,
E.
,
2011
, “
A Robust and Sensitive Metric for Quantifying Movement Smoothness
,”
IEEE Trans. Biomed. Eng.
,
59
(
8
), pp.
2126
2136
.
23.
Ryu
,
D.
,
Song
,
J.-B.
,
Kang
,
S.
, and
Kim
,
M.
,
2008
, “
Frequency Domain Stability Observer and Active Damping Control for Stable Haptic Interaction
,”
IET Control Theory Appl.
,
2
(
4
), pp.
261
268
.
24.
Dimeas
,
F.
, and
Aspragathos
,
N.
,
2016
, “
Online Stability in Human-Robot Cooperation With Admittance Control
,”
IEEE Trans. Haptic.
,
9
(
2
), pp.
267
278
.
25.
Roy
,
R. N.
,
Drougard
,
N.
,
Gateau
,
T.
,
Dehais
,
F.
, and
Chanel
,
C. P.
,
2020
, “
How Can Physiological Computing Benefit Human-Robot Interaction?
,”
Robotics
,
9
(
4
), p.
100
.
26.
Lele
,
P.
,
Sinclair
,
D.
, and
Weddell
,
G.
,
1954
, “
The Reaction Time to Touch
,”
J. Physiol.
,
123
(
1
), p.
187
203
.
27.
Salvendy
,
G.
,
2012
,
Handbook of Human Factors and Ergonomics
,
John Wiley & Sons
,
Hoboken, NJ
.
28.
Memar
,
A. H.
, and
Esfahani
,
E. T.
,
2018
, “
EEG Correlates of Motor Control Difficulty in Physical Human-Robot Interaction: A Frequency Domain Analysis
,”
2018 IEEE Haptics Symposium (HAPTICS)
,
San Francisco, CA
,
Mar. 25–28
, IEEE, pp.
229
234
.
29.
Lin
,
C.-L.
,
Shaw
,
F.-Z.
,
Young
,
K.-Y.
,
Lin
,
C.-T.
, and
Jung
,
T.-P.
,
2012
, “
EEG Correlates of Haptic Feedback in a Visuomotor Tracking Task
,”
NeuroImage
,
60
(
4
), pp.
2258
2273
.
30.
Peng
,
J.
,
Zikereya
,
T.
,
Shao
,
Z.
, and
Shi
,
K.
,
2024
, “
The Neuromechanical of Beta-Band Corticomuscular Coupling Within the Human Motor System
,”
Front. Neurosci.
,
18
, p.
1441002
.
31.
Haufe
,
S.
,
Treder
,
M. S.
,
Gugler
,
M. F.
,
Sagebaum
,
M.
,
Curio
,
G.
, and
Blankertz
,
B.
,
2011
, “
EEG Potentials Predict Upcoming Emergency Brakings During Simulated Driving
,”
J. Neural. Eng.
,
8
(
5
), p.
056001
.
32.
Liang
,
X.
,
Yu
,
Y.
,
Liu
,
Y.
,
Liu
,
K.
,
Liu
,
Y.
, and
Zhou
,
Z.
,
2023
, “
EEG-Based Emergency Braking Intention Detection During Simulated Driving
,”
BioMed. Eng. Online
,
22
(
1
), p.
65
.
33.
Nacpil
,
E. J. C.
,
Wang
,
Z.
,
Guan
,
M.
,
Nakano
,
K.
, and
Jeon
,
I.
,
2023
, “
EEG-Based Emergency Braking Prediction Using Data Ablation and SVM Classification
,”
IEEE Sens. J.
,
23
(
14
), pp.
16013
16019
.
34.
Teng
,
T.
,
Bi
,
L.
, and
Liu
,
Y.
,
2017
, “
EEG-Based Detection of Driver Emergency Braking Intention for Brain-Controlled Vehicles
,”
IEEE Trans. Intell. Trans. Syst.
,
19
(
6
), pp.
1766
1773
.
35.
Sharkawy
,
A.-N.
, and
Koustoumpardis
,
P. N.
,
2022
, “
Human–Robot Interaction: A Review and Analysis on Variable Admittance Control, Safety, and Perspectives
,”
Machines
,
10
(
7
), p.
591
.
36.
Gramfort
,
A.
,
Luessi
,
M.
,
Larson
,
E.
,
Engemann
,
D. A.
,
Strohmeier
,
D.
,
Brodbeck
,
C.
,
Goj
,
R.
, et al.
,
2013
, “
MEG and EEG Data Analysis With MNE-Python
,”
Front. Neurosci.
,
7
, p.
267
.
37.
Perreault
,
E. J.
,
Kirsch
,
R. F.
, and
Crago
,
P. E.
,
2001
, “
Effects of Voluntary Force Generation on the Elastic Components of Endpoint Stiffness
,”
Exp. Brain Res.
,
141
(
3
), pp.
312
323
.
38.
Gasser
,
T.
,
Bacher
,
P.
, and
Mocks
,
J.
,
1982
, “
Transformations Towards the Normal Distribution of Broad Band Spectral Parameters of the EEG
,”
Electroencephalogr. Clin. Neurophysiol.
,
53
(
1
), pp.
119
124
.
39.
Ng
,
A. Y.
,
2004
, “
Feature Selection, L 1 vs. L 2 Regularization, and Rotational Invariance
,”
Proceedings of the Twenty-First International Conference on Machine Learning
,
Banff, AB, Canada
,
July 4–8
, p.
78
.
40.
Schirrmeister
,
R. T.
,
Springenberg
,
J. T.
,
Fiederer
,
L. D. J.
,
Glasstetter
,
M.
,
Eggensperger
,
K.
,
Tangermann
,
M.
,
Hutter
,
F.
,
Burgard
,
W.
, and
Ball
,
T.
,
2017
, “
Deep Learning With Convolutional Neural Networks for EEG Decoding and Visualization
,”
Hum. Brain Mapp.
,
38
(
11
), pp.
5391
5420
.
41.
Passalis
,
N.
,
Tefas
,
A.
,
Kanniainen
,
J.
,
Gabbouj
,
M.
, and
Iosifidis
,
A.
,
2019
, “
Deep Adaptive Input Normalization for Time Series Forecasting
,”
IEEE Trans. Neural Netw. Learn. Syst.
,
31
(
9
), pp.
3760
3765
.
42.
Jujjavarapu
,
S. S.
,
Memar
,
A. H.
,
Karami
,
M. A.
, and
Esfahani
,
E. T.
,
2019
, “
Variable Stiffness Mechanism for Suppressing Unintended Forces in Physical Human–Robot Interaction
,”
ASME J. Mech. Rob.
,
11
(
2
), p.
020915
.
43.
Rocon
,
E.
, and
Pons
,
J. L.
,
2011
,
Exoskeletons in Rehabilitation Robotics: Tremor Suppression
, Vol.
69
.
Springer
,
New York
.