Abstract

In this study, we developed an offline, hierarchical intent recognition system for inferring the timing and direction of motion intent of a human operator when operating in an unstructured environment. There has been an increasing demand for robot agents to assist in these dynamic, rapid motions that are constantly evolving and require quick, accurate estimation of a user’s direction of travel. An experiment was conducted in a motion capture space with six subjects performing threat evasion in eight directions, and their mechanical and neuromuscular signals were recorded for use in our intent recognition system (XGBoost). Investigated against current, analytical methods, our system demonstrated superior performance with quicker direction of travel estimation occurring 140 ms earlier in the movement and a 11.6 deg reduction of error. The results showed that we could also predict the start of the movement 100 ms prior to the actual, thus allowing any physical systems to start up. Our direction estimation had an optimal performance of 8.8 deg, or 2.4% of the 360 deg range of travel, using three-axis kinetic data. The performance of other sensors and their combinations indicate that there are additional possibilities to obtain low estimation error. These findings are promising as they can be used to inform the design of a wearable robot aimed at assisting users in dynamic motions, while in environments with oncoming threats.

References

1.
Goodrich
,
M. A.
, and
Schultz
,
A. C.
,
2008
,
Human-Robot Interaction: A Survey
,
Now Publishers Inc
.,
Hanover, MA
.
2.
Young
,
A. J.
, and
Ferris
,
D. P.
,
2016
, “
State of the Art and Future Directions for Lower Limb Robotic Exoskeletons
,”
IEEE. Trans. Neural. Syst. Rehabil. Eng.
,
25
(
2
), pp.
171
182
.
3.
Hirai
,
K.
,
Hirose
,
M.
,
Haikawa
,
Y.
, and
Takenaka
,
T.
,
1998
, “
The Development of Honda Humanoid Robot
,”
1998 IEEE International Conference on Robotics and Automation (Cat. No. 98CH36146)
,
Leuven, Belgium
,
May 20
.
4.
Halilaj
,
E.
,
Rajagopal
,
A.
,
Fiterau
,
M.
,
Hicks
,
J. L.
,
Hastie
,
T. J.
, and
Delp
,
S. L.
,
2018
, “
Machine Learning in Human Movement Biomechanics: Best Practices, Common Pitfalls, and New Opportunities
,”
J. Biomech.
,
81
, pp.
1
11
.
5.
Huo
,
W.
,
Mohammed
,
S.
,
Moreno
,
J. C.
, and
Amirat
,
Y.
,
2014
, “
Lower Limb Wearable Robots for Assistance and Rehabilitation: A State of the Art
,”
IEEE Syst. J.
,
10
(
3
), pp.
1068
1081
.
6.
Kuo
,
C. H.
,
2016
, “
Trajectory and Heading Tracking of a Mecanum Wheeled Robot Using Fuzzy Logic Control
,”
2016 International Conference on Instrumentation, Control and Automation (ICA)
,
Bandung, Indonesia
,
Aug. 29–31
, IEEE, pp.
54
59
.
7.
Erden
,
M. S.
, and
Tomiyama
,
T.
,
2010
, “
Human-Intent Detection and Physically Interactive Control of a Robot Without Force Sensors
,”
IEEE Trans. Rob.
,
26
(
2
), pp.
370
382
.
8.
Jung
,
S.
,
Hsia
,
T. C.
, and
Bonitz
,
R. G.
,
2004
, “
Force Tracking Impedance Control of Robot Manipulators Under Unknown Environment
,”
IEEE Trans. Control Syst. Technol.
,
12
(
3
), pp.
474
483
.
9.
Aguirre-Ollinger
,
G.
,
Colgate
,
J. E.
,
Peshkin
,
M. A.
, and
Goswami
,
A.
,
2012
, “
Inertia Compensation Control of a One-Degree-of-Freedom Exoskeleton for Lower-Limb Assistance: Initial Experiments
,”
IEEE. Trans. Neural. Syst. Rehabil. Eng.
,
20
(
1
), pp.
68
77
.
10.
Croft
,
D.
,
2003
, “
Estimating Intent for Human-Robot Interaction
,”
IEEE International Conference on Advanced Robotics
,
Coimbra, Portugal
,
June 30–July 3
, pp.
810
815
.
11.
Yang
,
S. X.
, and
Meng
,
M.
,
2000
, “
An Efficient Neural Network Approach to Dynamic Robot Motion Planning
,”
Neural Netw.
,
13
(
2
), pp.
143
148
.
12.
Varol
,
H. A.
,
Sup
,
F.
, and
Goldfarb
,
M.
,
2009
, “
Multiclass Real-Time Intent Recognition of a Powered Lower Limb Prosthesis
,”
IEEE Trans. Biomed. Eng.
,
57
(
3
), pp.
542
551
.
13.
Begg
,
R.
, and
Kamruzzaman
,
J.
,
2005
, “
A Machine Learning Approach for Automated Recognition of Movement Patterns Using Basic, Kinetic and Kinematic Gait Data
,”
J. Biomech.
,
38
(
3
), pp.
401
408
.
14.
Young
,
A. J.
,
Simon
,
A. M.
,
Fey
,
N. P.
, and
Hargrove
,
L. J.
,
2014
, “
Intent Recognition in a Powered Lower Limb Prosthesis Using Time History Information
,”
Ann. Biomed. Eng.
,
42
(
3
), pp.
631
641
.
15.
Huang
,
H.
,
Zhang
,
F.
,
Hargrove
,
L. J.
,
Dou
,
Z.
,
Rogers
,
D. R.
, and
Englehart
,
K. B.
,
2011
, “
Continuous Locomotion-Mode Identification for Prosthetic Legs Based on Neuromuscular-Mechanical Fusion
,”
IEEE Trans. Biomed. Eng.
,
58
(
10
), pp.
2867
2875
.
16.
Young
,
A.
,
Kuiken
,
T.
, and
Hargrove
,
L.
,
2014
, “
Analysis of Using EMG and Mechanical Sensors to Enhance Intent Recognition in Powered Lower Limb Prostheses
,”
J. Neural. Eng.
,
11
(
5
), p.
056021
.
17.
Criminisi
,
A.
,
Shotton
,
J.
, and
Konukoglu
,
E.
,
2012
, “
Decision Forests: A Unified Framework for Classification, Regression, Density Estimation, Manifold Learning and Semi-Supervised Learning
,”
Found. Trends® Comput. Graphics Vision
,
7
(
2–3
), pp.
81
227
.
18.
Gregory
,
U.
, and
Ren
,
L.
,
2019
, “
Intent Prediction of Multi-Axial Ankle Motion Using Limited EMG Signals
,”
Front. Bioeng. Biotech.
,
7
, p.
335
.
19.
Joshi
,
D.
, and
Hahn
,
M. E.
,
2016
, “
Terrain and Direction Classification of Locomotion Transitions Using Neuromuscular and Mechanical Input
,”
Ann. Biomed. Eng.
,
44
(
4
), pp.
1275
1284
.
20.
Irastorza-Landa
,
N.
,
Sarasola-Sanz
,
A.
,
López-Larraz
,
E.
,
Bibián
,
C.
,
Shiman
,
F.
,
Birbaumer
,
N.
, and
Ramos-Murguialday
,
A.
,
2017
, “
Design of Continuous EMG Classification Approaches Towards the Control of a Robotic Exoskeleton in Reaching Movements
,”
2017 International Conference on Rehabilitation Robotics (ICORR)
,
London, UK
,
July 17–20
, IEEE, pp.
128
133
.
21.
Rand
,
M. K.
, and
Ohtsuki
,
T.
,
2000
, “
EMG Analysis of Lower Limb Muscles in Humans During Quick Change in Running Directions
,”
Gait Post.
,
12
(
2
), pp.
169
183
.
22.
Artemiadis
,
P. K.
, and
Kyriakopoulos
,
K. J.
,
2009
, “EMG-Based Position and Force Estimates in Coupled Human-Robot Systems: Towards EMG-Controlled Exoskeletons”
Experimental Robotics
, Springer Tracts in Advanced Robotics, Vol. 54,
O.
Khatib
,
V.
Kumar
, and
G. J.
Pappas
, eds.,
Springer
,
Berlin/Heidelberg
, pp.
241
250
.
23.
Young
,
A. J.
,
Smith
,
L. H.
,
Rouse
,
E. J.
, and
Hargrove
,
L. J.
,
2012
, “
Classification of Simultaneous Movements Using Surface EMG Pattern Recognition
,”
IEEE Trans. Biomed. Eng.
,
60
(
5
), pp.
1250
1258
.
24.
Accogli
,
A.
,
Grazi
,
L.
,
Crea
,
S.
,
Panarese
,
A.
,
Carpaneto
,
J.
,
Vitiello
,
N.
, and
Micera
,
S.
,
2017
, “EMG-Based Detection of User’s Intentions for Human-Machine Shared Control of an Assistive Upper-Limb Exoskeleton,”
Wearable Robotics: Challenges and Trends
, Biosystems & Biorobotics, vol. 16,
J.
González-Vargas
J.
Ibáñez
,
J.
Contreras-Vidal
,
H.
van der Kooij
, and
J.
Pons
, eds.,
Springer
,
Cham
, Vol. 16, pp.
181
185
.
25.
Havens
,
K. L.
, and
Sigward
,
S. M.
,
2015
, “
Joint and Segmental Mechanics Differ Between Cutting Maneuvers in Skilled Athletes
,”
Gait Post.
,
41
(
1
), pp.
33
38
.
26.
Koopman
,
B.
,
Grootenboer
,
H. J.
, and
De Jongh
,
H. J.
,
1995
, “
An Inverse Dynamics Model for the Analysis, Reconstruction and Prediction of Bipedal Walking
,”
J. Biomech.
,
28
(
11
), pp.
1369
1376
.
27.
Kawamori
,
N.
,
Newton
,
R.
, and
Nosaka
,
K.
,
2014
, “
Effects of Weighted Sled Towing on Ground Reaction Force During the Acceleration Phase of Sprint Running
,”
J. Sports Sci.
,
32
(
12
), pp.
1139
1145
.
28.
James
,
C. R.
,
Sizer
,
P. S.
,
Starch
,
D. W.
,
Lockhart
,
T. E.
, and
Slauterbeck
,
J.
,
2004
, “
Gender Differences Among Sagittal Plane Knee Kinematic and Ground Reaction Force Characteristics During a Rapid Sprint and Cut Maneuver
,”
Res. Q. Exercise Sport
,
75
(
1
), pp.
31
38
.
29.
Adamczyk
,
P. G.
, and
Kuo
,
A. D.
,
2009
, “
Redirection of Center-of-Mass Velocity During the Step-to-Step Transition of Human Walking
,”
J. Exp. Biol.
,
212
(
16
), pp.
2668
2678
.
30.
Yeo
,
S. S.
, and
Park
,
G. Y.
,
2020
, “
Accuracy Verification of Spatio-Temporal and Kinematic Parameters for Gait Using Inertial Measurement Unit System
,”
Sensors
,
20
(
5
), p.
1343
.
31.
Fong
,
D. T.-P.
, and
Chan
,
Y.-Y.
,
2010
, “
The Use of Wearable Inertial Motion Sensors in Human Lower Limb Biomechanics Studies: A Systematic Review
,”
Sensors
,
10
(
12
), pp.
11556
11565
.
32.
Rainoldi
,
A.
,
Melchiorri
,
G.
, and
Caruso
,
I.
,
2004
, “
A Method for Positioning Electrodes During Surface EMG Recordings in Lower Limb Muscles
,”
J. Neurosci. Methods.
,
134
(
1
), pp.
37
43
.
33.
Kim
,
S.
, and
Nussbaum
,
M.
,
2014
, “
Evaluation of Two Approaches for Aligning Data Obtained From a Motion Capture System and an In-Shoe Pressure Measurement System
,”
Sensors
,
14
(
9
), pp.
16994
17007
.
34.
Delp
,
S. L.
,
Anderson
,
F. C.
,
Arnold
,
A. S.
,
Loan
,
P.
,
Habib
,
A.
,
John
,
C. T.
,
Guendelman
,
E.
, and
Thelen
,
D. G.
,
2007
, “
Opensim: Open-Source Software to Create and Analyze Dynamic Simulations of Movement
,”
IEEE Trans. Biomed. Eng.
,
54
(
11
), pp.
1940
1950
.
35.
Jain
,
A.
, and
Zongker
,
D.
,
1997
, “
Feature Selection: Evaluation, Application, and Small Sample Performance
,”
IEEE. Trans. Pattern. Anal. Mach. Intell.
,
19
(
2
), pp.
153
158
.
36.
Phinyomark
,
A.
,
Limsakul
,
C.
, and
Phukpattaranont
,
P.
,
2009
, “
A Novel Feature Extraction for Robust EMG Pattern Recognition
,”
J. Comput.
,
1
, pp.
71
80
.
37.
Bhakta
,
K.
,
Camargo
,
J.
,
Donovan
,
L.
,
Herrin
,
K.
, and
Young
,
A.
,
2020
, “
Machine Learning Model Comparisons of User Independent & Dependent Intent Recognition Systems for Powered Prostheses
,”
IEEE Rob. Auto. Lett.
,
5
(
4
), pp.
5393
5400
.
38.
Chen
,
T.
, and
Guestrin
,
C.
,
2016
, “
Xgboost: A Scalable Tree Boosting System
,”
Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining
,
San Francisco, CA
,
Aug. 13–17
.
39.
Dos’Santos
,
T.
,
Thomas
,
C.
,
Comfort
,
P.
, and
Jones
,
P. A.
,
2018
, “
The Effect of Angle and Velocity on Change of Direction Biomechanics: An Angle-Velocity Trade-Off
,”
Sports Med.
,
48
(
10
), pp.
2235
2253
.
40.
Yu
,
Z.
, and
Lee
,
M.
,
2015
, “
Human Motion Based Intent Recognition Using a Deep Dynamic Neural Model
,”
Rob. Auton. Syst.
,
71
, pp.
134
149
.
41.
Bi
,
L.
,
Feleke
,
A. G.
, and
Guan
,
C.
,
2019
, “
A Review on EMG-Based Motor Intention Prediction of Continuous Human Upper Limb Motion for Human-Robot Collaboration
,”
Biomed. Signal Process. Control
,
51
, pp.
113
127
.
42.
Dejnabadi
,
H.
,
Jolles
,
B. M.
,
Casanova
,
E.
,
Fua
,
P.
, and
Aminian
,
K.
,
2006
, “
Estimation and Visualization of Sagittal Kinematics of Lower Limbs Orientation Using Body-Fixed Sensors
,”
IEEE Trans. Biomed. Eng.
,
53
(
7
), pp.
1385
1393
.
43.
Harms
,
H.
,
Amft
,
O.
,
Winkler
,
R.
,
Schumm
,
J.
,
Kusserow
,
M.
, and
Tröster
,
G.
,
2010
, “
Ethos: Miniature Orientation Sensor for Wearable Human Motion Analysis
,”
SENSORS, 2010 IEEE
,
Waikoloa, HI
,
Nov. 1–4
, pp.
1037
1042
.
44.
Pappas
,
I. P.
,
Keller
,
T.
,
Mangold
,
S.
,
Popovic
,
M. R.
,
Dietz
,
V.
, and
Morari
,
M.
,
2004
, “
A Reliable Gyroscope-Based Gait-Phase Detection Sensor Embedded in a Shoe Insole
,”
IEEE Sens. J.
,
4
(
2
), pp.
268
274
.
45.
Rosquist
,
P. G.
,
Collins
,
G.
,
Merrell
,
A. J.
,
Tuttle
,
N. J.
,
Tracy
,
J. B.
,
Bird
,
E. T.
,
Seeley
,
M. K.
,
Fullwood
,
D. T.
,
Christensen
,
W. F.
, and
Bowden
,
A. E.
,
2017
, “
Estimation of 3d Ground Reaction Force Using Nanocomposite Piezo-Responsive Foam Sensors During Walking
,”
Ann. Biomed. Eng.
,
45
(
9
), pp.
2122
2134
.
46.
Lincoln
,
L. S.
,
Bamberg
,
S. J. M.
,
Parsons
,
E.
,
Salisbury
,
C.
, and
Wheeler
,
J.
,
2012
, “
An Elastomeric Insole for 3-Axis Ground Reaction Force Measurement
,”
2012 4th IEEE RAS & EMBS International Conference on Biomedical Robotics and Biomechatronics (BioRob)
,
Rome, Italy
,
Aug. 31
, IEEE, pp.
1512
1517
.
You do not currently have access to this content.