Abstract

Improper controller parameter settings in physical human–robot interaction (pHRI) can lead to instability, compromising both safety and system performance. This study investigates the relationship between cognitive and physical aspects of co-manipulation by leveraging electroencephalography (EEG) to predict instability in physical human–robot interaction. Using elastic net regression and deep convolutional neural networks, we estimate instability as subjects guide a robot through predefined trajectories under varying admittance control settings. Our results show that EEG signals can predict instability up to 2 s before it manifests in force data. Moreover, the deep learning-based approach significantly outperforms elastic regression, achieving a notable (∼10%) improvement in predicting the instability index. These findings highlight the potential of EEG-based monitoring for enhancing real-time stability assessment in pHRI.

References

1.
Goodrich
,
M. A.
, and
Schultz
,
A. C.
,
2008
, “
Human–Robot Interaction: A Survey
,”
Found. Trends Hum. Comput. Interact.
,
1
(
3
), pp.
203
275
.
2.
De Santis
,
A.
,
Siciliano
,
B.
,
De Luca
,
A.
, and
Bicchi
,
A.
,
2008
, “
An Atlas of Physical Human–Robot Interaction
,”
Mech. Mach. Theory
,
43
(
3
), pp.
253
270
.
3.
Villani
,
V.
,
Pini
,
F.
,
Leali
,
F.
, and
Secchi
,
C.
,
2018
, “
Survey on Human–Robot Collaboration in Industrial Settings: Safety, Intuitive Interfaces and Applications
,”
Mechatronics
,
55
, pp.
248
266
.
4.
Sirintuna
,
D.
,
Ozdamar
,
I.
,
Aydin
,
Y.
, and
Basdogan
,
C.
,
2020
, “
Detecting Human Motion Intention During pHRI Using Artificial Neural Networks Trained by EMG Signals
,”
2020 29th IEEE International Conference on Robot and Human Interactive Communication (RO-MAN)
,
Naples, Italy
,
Aug. 31–Sept. 4
, IEEE, pp.
1280
1287
.
5.
Xing
,
B.
, and
Marwala
,
T.
,
2018
, “Introduction to Human Robot Interaction,”
Smart Maintenance for Human–Robot Interaction
,
B.
Xing
, and
T.
Marwala
, eds.,
Springer
,
New York
, pp.
3
19
.
6.
Mutlu
,
B.
,
Roy
,
N.
, and
Sabanovic
,
S.
,
2016
, “Cognitive Human–Robot Interaction,”
Springer Handbook of Robotics
,
B.
Siciliano
, and
O.
Khatib
, eds.,
Springer
,
New York
, pp.
1907
1934
.
7.
Memar
,
A.
, and
Esfahani
,
E.
,
2018
, “
Using EEG for Predicting User Preferences of Physical Compliance in Human-Robot Cooperation
,”
2018 2nd International Neuroergonomics Conference
,
Philadelphia, PA
,
June 27–29
, pp.
1
2
.
8.
Mashayekhi
,
M.
, and
Moghaddam
,
M. M.
,
2022
, “
EMG-Driven Fatigue-Based Self-Adapting Admittance Control of a Hand Rehabilitation Robot
,”
J. Biomech.
,
138
, p.
111104
.
9.
Distefano
,
J. P.
,
Chowdhury
,
S.
, and
Esfahani
,
E.
,
2024
, “
Exploring Human-Swarm Interaction Dynamics in Cyber-Physical Systems: A Physiological Approach
,”
J. Integr. Des. Process Sci.
, p.
10920617241292155
.
10.
Li
,
S.
,
Zheng
,
P.
,
Liu
,
S.
,
Wang
,
Z.
,
Wang
,
X. V.
,
Zheng
,
L.
, and
Wang
,
L.
,
2023
, “
Proactive Human–Robot Collaboration: Mutual-Cognitive, Predictable, and Self-Organising Perspectives
,”
Rob. Comput.-Integr. Manuf.
,
81
, p.
102510
.
11.
Rubagotti
,
M.
,
Tusseyeva
,
I.
,
Baltabayeva
,
S.
,
Summers
,
D.
, and
Sandygulova
,
A.
,
2022
, “
Perceived Safety in Physical Human–Robot Interaction-A Survey
,”
Rob. Auton. Syst.
,
151
, p.
104047
.
12.
Manjunatha
,
H.
,
Pareek
,
S.
,
Memar
,
A. H.
,
Kesavadas
,
T.
, and
Esfahani
,
E. T.
,
2020
, “
Effect of Haptic Assistance Strategy on Mental Engagement in Fine Motor Tasks
,”
J. Med. Rob. Res.
,
5
(
01n02
), p.
2041004
.
13.
Calanca
,
A.
,
Muradore
,
R.
, and
Fiorini
,
P.
,
2015
, “
A Review of Algorithms for Compliant Control of Stiff and Fixed-Compliance Robots
,”
IEEE/ASME Trans. Mechatron.
,
21
(
2
), pp.
613
624
.
14.
Djuric
,
A. M.
,
Urbanic
,
R.
, and
Rickli
,
J.
,
2016
, “
A Framework for Collaborative Robot (Cobot) Integration in Advanced Manufacturing Systems
,”
SAE Int. J. Mater. Manuf.
,
9
(
2
), pp.
457
464
.
15.
Hu
,
M.
,
2023
, “
Research on Safety Design and Optimization of Collaborative Robots
,”
Int. J. Intell. Rob. Appl.
,
7
(
4
), pp.
795
809
.
16.
Keemink
,
A. Q.
,
van der Kooij
,
H.
, and
Stienen
,
A. H.
,
2018
, “
Admittance Control for Physical Human–Robot Interaction
,”
Int. J. Rob. Res.
,
37
(
11
), pp.
1421
1444
.
17.
Memar
,
A. H.
, and
Esfahani
,
E. T.
,
2019
, “
Objective Assessment of Human Workload in Physical Human-Robot Cooperation Using Brain Monitoring
,”
ACM Trans. Hum.-Rob Interact.
,
9
(
2
), pp.
1
21
.
18.
Hemanth
,
M.
,
Jujjavarapu
,
S. S.
, and
Esfahani
,
E. T.
,
2020
, “
Classification of Motor Control Difficulty Using EMG in Physical Human-Robot Interaction
,”
2020 IEEE International Conference on Systems, Man, and Cybernetics (SMC)
,
Toronto, ON, Canada
,
Oct. 7–10
, IEEE, pp.
2708
2713
.
19.
Balasubramanian
,
S.
,
Melendez-Calderon
,
A.
,
Roby-Brami
,
A.
, and
Burdet
,
E.
,
2015
, “
On the Analysis of Movement Smoothness
,”
J. Neuroeng. Rehabil.
,
12
(
1
), p.
112
.
20.
Teo
,
C. L.
,
Burdet
,
E.
, and
Lim
,
H.
,
2002
, “
A Robotic Teacher of Chinese Handwriting
,”
Proceedings 10th Symposium on Haptic Interfaces for Virtual Environment and Teleoperator Systems (HAPTICS 2002)
,
Orlando, FL
,
Mar. 24–25
, IEEE, pp.
335
341
.
21.
Vaisman
,
L.
,
Dipietro
,
L.
, and
Krebs
,
H. I.
,
2012
, “
A Comparative Analysis of Speed Profile Models for Wrist Pointing Movements
,”
IEEE Trans. Neural Syst. Rehabil. Eng.
,
21
(
5
), pp.
756
766
.
22.
Balasubramanian
,
S.
,
Melendez-Calderon
,
A.
, and
Burdet
,
E.
,
2011
, “
A Robust and Sensitive Metric for Quantifying Movement Smoothness
,”
IEEE Trans. Biomed. Eng.
,
59
(
8
), pp.
2126
2136
.
23.
Ryu
,
D.
,
Song
,
J.-B.
,
Kang
,
S.
, and
Kim
,
M.
,
2008
, “
Frequency Domain Stability Observer and Active Damping Control for Stable Haptic Interaction
,”
IET Control Theory Appl.
,
2
(
4
), pp.
261
268
.
24.
Dimeas
,
F.
, and
Aspragathos
,
N.
,
2016
, “
Online Stability in Human-Robot Cooperation With Admittance Control
,”
IEEE Trans. Haptic.
,
9
(
2
), pp.
267
278
.
25.
Roy
,
R. N.
,
Drougard
,
N.
,
Gateau
,
T.
,
Dehais
,
F.
, and
Chanel
,
C. P.
,
2020
, “
How Can Physiological Computing Benefit Human-Robot Interaction?
,”
Robotics
,
9
(
4
), p.
100
.
26.
Lele
,
P.
,
Sinclair
,
D.
, and
Weddell
,
G.
,
1954
, “
The Reaction Time to Touch
,”
J. Physiol.
,
123
(
1
), p.
187
203
.
27.
Salvendy
,
G.
,
2012
,
Handbook of Human Factors and Ergonomics
,
John Wiley & Sons
,
Hoboken, NJ
.
28.
Memar
,
A. H.
, and
Esfahani
,
E. T.
,
2018
, “
EEG Correlates of Motor Control Difficulty in Physical Human-Robot Interaction: A Frequency Domain Analysis
,”
2018 IEEE Haptics Symposium (HAPTICS)
,
San Francisco, CA
,
Mar. 25–28
, IEEE, pp.
229
234
.
29.
Lin
,
C.-L.
,
Shaw
,
F.-Z.
,
Young
,
K.-Y.
,
Lin
,
C.-T.
, and
Jung
,
T.-P.
,
2012
, “
EEG Correlates of Haptic Feedback in a Visuomotor Tracking Task
,”
NeuroImage
,
60
(
4
), pp.
2258
2273
.
30.
Peng
,
J.
,
Zikereya
,
T.
,
Shao
,
Z.
, and
Shi
,
K.
,
2024
, “
The Neuromechanical of Beta-Band Corticomuscular Coupling Within the Human Motor System
,”
Front. Neurosci.
,
18
, p.
1441002
.
31.
Haufe
,
S.
,
Treder
,
M. S.
,
Gugler
,
M. F.
,
Sagebaum
,
M.
,
Curio
,
G.
, and
Blankertz
,
B.
,
2011
, “
EEG Potentials Predict Upcoming Emergency Brakings During Simulated Driving
,”
J. Neural. Eng.
,
8
(
5
), p.
056001
.
32.
Liang
,
X.
,
Yu
,
Y.
,
Liu
,
Y.
,
Liu
,
K.
,
Liu
,
Y.
, and
Zhou
,
Z.
,
2023
, “
EEG-Based Emergency Braking Intention Detection During Simulated Driving
,”
BioMed. Eng. Online
,
22
(
1
), p.
65
.
33.
Nacpil
,
E. J. C.
,
Wang
,
Z.
,
Guan
,
M.
,
Nakano
,
K.
, and
Jeon
,
I.
,
2023
, “
EEG-Based Emergency Braking Prediction Using Data Ablation and SVM Classification
,”
IEEE Sens. J.
,
23
(
14
), pp.
16013
16019
.
34.
Teng
,
T.
,
Bi
,
L.
, and
Liu
,
Y.
,
2017
, “
EEG-Based Detection of Driver Emergency Braking Intention for Brain-Controlled Vehicles
,”
IEEE Trans. Intell. Trans. Syst.
,
19
(
6
), pp.
1766
1773
.
35.
Sharkawy
,
A.-N.
, and
Koustoumpardis
,
P. N.
,
2022
, “
Human–Robot Interaction: A Review and Analysis on Variable Admittance Control, Safety, and Perspectives
,”
Machines
,
10
(
7
), p.
591
.
36.
Gramfort
,
A.
,
Luessi
,
M.
,
Larson
,
E.
,
Engemann
,
D. A.
,
Strohmeier
,
D.
,
Brodbeck
,
C.
,
Goj
,
R.
, et al.
,
2013
, “
MEG and EEG Data Analysis With MNE-Python
,”
Front. Neurosci.
,
7
, p.
267
.
37.
Perreault
,
E. J.
,
Kirsch
,
R. F.
, and
Crago
,
P. E.
,
2001
, “
Effects of Voluntary Force Generation on the Elastic Components of Endpoint Stiffness
,”
Exp. Brain Res.
,
141
(
3
), pp.
312
323
.
38.
Gasser
,
T.
,
Bacher
,
P.
, and
Mocks
,
J.
,
1982
, “
Transformations Towards the Normal Distribution of Broad Band Spectral Parameters of the EEG
,”
Electroencephalogr. Clin. Neurophysiol.
,
53
(
1
), pp.
119
124
.
39.
Ng
,
A. Y.
,
2004
, “
Feature Selection, L 1 vs. L 2 Regularization, and Rotational Invariance
,”
Proceedings of the Twenty-First International Conference on Machine Learning
,
Banff, AB, Canada
,
July 4–8
, p.
78
.
40.
Schirrmeister
,
R. T.
,
Springenberg
,
J. T.
,
Fiederer
,
L. D. J.
,
Glasstetter
,
M.
,
Eggensperger
,
K.
,
Tangermann
,
M.
,
Hutter
,
F.
,
Burgard
,
W.
, and
Ball
,
T.
,
2017
, “
Deep Learning With Convolutional Neural Networks for EEG Decoding and Visualization
,”
Hum. Brain Mapp.
,
38
(
11
), pp.
5391
5420
.
41.
Passalis
,
N.
,
Tefas
,
A.
,
Kanniainen
,
J.
,
Gabbouj
,
M.
, and
Iosifidis
,
A.
,
2019
, “
Deep Adaptive Input Normalization for Time Series Forecasting
,”
IEEE Trans. Neural Netw. Learn. Syst.
,
31
(
9
), pp.
3760
3765
.
42.
Jujjavarapu
,
S. S.
,
Memar
,
A. H.
,
Karami
,
M. A.
, and
Esfahani
,
E. T.
,
2019
, “
Variable Stiffness Mechanism for Suppressing Unintended Forces in Physical Human–Robot Interaction
,”
ASME J. Mech. Rob.
,
11
(
2
), p.
020915
.
43.
Rocon
,
E.
, and
Pons
,
J. L.
,
2011
,
Exoskeletons in Rehabilitation Robotics: Tremor Suppression
, Vol.
69
.
Springer
,
New York
.
You do not currently have access to this content.