Due to various applications of human motion capture techniques, developing low-cost methods that would be applicable in nonlaboratory environments is under consideration. MEMS inertial sensors and Kinect are two low-cost devices that can be utilized in home-based motion capture systems, e.g., home-based rehabilitation. In this work, an unscented Kalman filter approach was developed based on the complementary properties of Kinect and the inertial sensors to fuse the orientation data of these two devices for human arm motion tracking during both stationary shoulder joint position and human body movement. A new measurement model of the fusion algorithm was obtained that can compensate for the inertial sensors drift problem in high dynamic motions and also joints occlusion in Kinect. The efficiency of the proposed algorithm was evaluated by an optical motion tracker system. The errors were reduced by almost 50% compared to cases when either inertial sensor or Kinect measurements were utilized.

References

References
1.
Zhou
,
H.
, and
Hu
,
H.
,
2008
, “
Human Motion Tracking for Rehabilitation—A Survey
,”
Biomed. Signal Process. Control
,
3
(
1
), pp.
1
18
.
2.
Bachmann
,
E. R.
,
McGhee
,
R. B.
,
Yun
,
X.
, and
Zyda
,
M. J.
,
2001
, “
Inertial and Magnetic Posture Tracking for Inserting Humans Into Networked Virtual Environments
,”
ACM Symposium on Virtual Reality Software and Technology
(
VRST '01
), Banff, AB, Canada, Nov. 15–17, pp.
9
16
.
3.
Zhang
,
Z.
,
2012
, “
Microsoft Kinect Sensor and Its Effect
,”
IEEE MultiMedia
,
19
(
2
), pp.
4
10
.
4.
Ahmadi
,
A.
,
Destelle
,
F.
,
Monaghan
,
D.
,
Moran
,
K.
,
O'Connor
,
N. E.
,
Unzueta
,
L.
, and
Linaza
,
M. T.
,
2015
, “
Human Gait Monitoring Using Body-Worn Inertial Sensors and Kinematic Modeling
,”
IEEE SENSORS
2015, Busan, South Korea, Nov. 1–4, pp.
540
547
.
5.
Zhang
,
Z.-Q.
, and
Wu
,
J.-K.
,
2011
, “
A Novel Hierarchical Information Fusion Method for Three-Dimensional Upper Limb Motion Estimation
,”
IEEE Trans. Instrum. Meas.
,
60
(
11
), pp.
3709
3719
.
6.
Tao
,
G.
,
Huang
,
Z.
,
Sun
,
Y.
,
Yao
,
S.
, and
Wu
,
J.
,
2013
, “
Biomechanical Model-Based Multi-Sensor Motion Estimation
,”
IEEE Sensors Applications Symposium
(
SAS
), Galveston, TX, Feb. 19–21, pp.
156
161
.
7.
El-Gohary
,
M.
,
Holmstrom
,
L.
,
Huisinga
,
J.
,
King
,
E.
,
McNames
,
J.
, and
Horak
,
F.
,
2011
, “
Upper Limb Joint Angle Tracking With Inertial Sensors
,”
Annual International Conference of the IEEE Engineering in Medicine and Biology Society
(
IEMBS
), Boston, MA, Aug. 30–Sept. 3, pp.
5629
5632
.
8.
Renaudin
,
V.
,
Afzal
,
M. H.
, and
Lachapelle
,
G.
,
2010
, “
Complete Triaxis Magnetometer Calibration in the Magnetic Domain
,”
J. Sens.
,
2010
, p.
967245
.
9.
Luinge
,
H. J.
, and
Veltink
,
P. H.
,
2005
, “
Measuring Orientation of Human Body Segments Using Miniature Gyroscopes and Accelerometers
,”
Med. Biol. Eng. Comput.
,
43
(
2
), pp.
273
282
.
10.
Le
,
T.-L.
,
Nguyen
,
M.-Q.
, and
Nguyen
,
T.-T.-M.
,
2013
, “
Human Posture Recognition Using Human Skeleton Provided by Kinect
,”
International Conference on Computing, Management, and Telecommunications
(
ComManTel
),
Ho Chi Minh City
,
Vietnam
, Jan. 21–24, pp.
340
345
.
11.
El-laithy
,
R. A.
,
Huang
,
J.
, and
Yeh
,
M.
,
2012
, “
Study on the Use of Microsoft Kinect for Robotics Applications
,”
IEEE/ION Position Location and Navigation Symposium
(
PLANS
), Myrtle Beach, SC, Apr. 23–26, pp.
1280
1288
.
12.
Gabel
,
M.
,
Gilad-Bachrach
,
R.
,
Renshaw
,
E.
, and
Schuster
,
A.
,
2012
, “
Full Body Gait Analysis With Kinect
,”
Annual International Conference of the IEEE Engineering in Medicine and Biology Society
(
EMBC
), San Deigo, CA, Aug. 28–Sept. 1, pp.
1964
1967
.
13.
Pfister
,
A.
,
West
,
A. M.
,
Bronner
,
S.
, and
Noah
,
J. A.
,
2014
, “
Comparative Abilities of Microsoft Kinect and Vicon 3D Motion Capture for Gait Analysis
,”
J. Med. Eng. Technol.
,
38
(
5
), pp.
274
280
.
14.
Clark
,
R. A.
,
Pua
,
Y.-H.
,
Fortin
,
K.
,
Ritchie
,
C.
,
Webster
,
K. E.
,
Denehy
,
L.
, and
Bryant
,
A. L.
,
2012
, “
Validity of the Microsoft Kinect for Assessment of Postural Control
,”
Gait Posture
,
36
(
3
), pp.
372
377
.
15.
Dutta
,
T.
,
2012
, “
Evaluation of the Kinect Sensor for 3-D Kinematic Measurement in the Workplace
,”
Appl. Ergon.
,
43
(
4
), pp.
645
649
.
16.
Webster
,
D.
, and
Celik
,
O.
,
2014
, “
Experimental Evaluation of Microsoft Kinect's Accuracy and Capture Rate for Stroke Rehabilitation Applications
,”
IEEE Haptics Symposium
(
HAPTICS
), Houston, TX, Feb. 23–26, pp.
455
460
.
17.
Bo
,
A.
,
Hayashibe
,
M.
, and
Poignet
,
P.
,
2011
, “
Joint Angle Estimation in Rehabilitation With Inertial Sensors and Its Integration With Kinect
,”
33rd Annual International Conference of the IEEE Engineering in Medicine and Biology Society
(
EMBC '11
), Boston, MA, Aug. 30–Sept. 3, pp.
3479
3483
.
18.
Feng
,
S.
, and
Murray-Smith
,
R.
,
2014
, “
Fusing Kinect Sensor and Inertial Sensors With Multi-Rate Kalman Filter
,”
IET Conference on Data Fusion & Target Tracking: Algorithms and Applications
(
DF&TT 2014
), Liverpool, UK, Apr. 30, pp.
1
8
.
19.
Tian
,
Y.
,
Meng
,
X.
,
Tao
,
D.
,
Liu
,
D.
, and
Feng
,
C.
,
2015
, “
Upper Limb Motion Tracking With the Integration of IMU and Kinect
,”
Neurocomputing
,
159
, pp.
207
218
.
20.
Destelle
,
F.
,
Ahmadi
,
A.
,
O'Connor
,
N. E.
,
Moran
,
K.
,
Chatzitofis
,
A.
,
Zarpalas
,
D.
, and
Daras
,
P.
,
2014
, “
Low-Cost Accurate Skeleton Tracking Based on Fusion of Kinect and Wearable Inertial Sensors
,”
22nd European Signal Processing Conference
(
EUSIPCO
), Lisbon, Portugal, Sept. 1–5, pp.
371
375
.
21.
Kalkbrenner
,
C.
,
Hacker
,
S.
,
Algorri
,
M.-E.
, and
Blechschmidt-Trapp
,
R.
,
2014
, “
Motion Capturing With Inertial Measurement Units and Kinect-Tracking of Limb Movement Using Optical and Orientation Information
,” International Conference on Biomedical Electronics and Devices (
BIODEVICES 2014
), Angers, France, Mar. 3–6, pp.
120
126
.
22.
Lizarralde
,
F.
, and
Wen
,
J. T.
,
1996
, “
Attitude Control Without Angular Velocity Measurement: A Passivity Approach
,”
IEEE Trans. Autom. Control
,
41
(
3
), pp.
468
472
.
23.
Yun
,
X.
,
Lizarraga
,
M.
,
Bachmann
,
E. R.
, and
McGhee
,
R. B.
,
2003
, “
An Improved Quaternion-Based Kalman Filter for Real-Time Tracking of Rigid Body Orientation
,”
IEEE/RSJ International Conference on Intelligent Robots and Systems
(
IROS 2003
), Las Vegas, NV, Oct. 27–31, Vol.
2
, pp.
1074
1079
.
24.
Simon
,
D.
,
2006
,
Optimal State Estimation: Kalman, H∞, and Nonlinear Approaches
,
Wiley
,
Hoboken, NJ
.
25.
Shuster
,
M. D.
, and
Oh
,
S.
,
1981
, “
Three-Axis Attitude Determination From Vector Observations
,”
J. Guid., Control, Dyn.
,
4
(
1
), pp.
70
77
.
26.
Yun
,
X.
,
Bachmann
,
E. R.
, and
McGhee
,
R. B.
,
2008
, “
A Simplified Quaternion-Based Algorithm for Orientation Estimation From Earth Gravity and Magnetic Field Measurements
,”
IEEE Trans. Instrum. Meas.
,
57
(
3
), pp.
638
650
.
27.
Sabatini
,
A. M.
,
2006
, “
Quaternion-Based Extended Kalman Filter for Determining Orientation by Inertial and Magnetic Sensing
,”
IEEE Trans. Biomed. Eng.
,
53
(
7
), pp.
1346
1356
.
28.
Julier
,
S. J.
, and
Uhlmann
,
J. K.
,
1997
, “
New Extension of the Kalman Filter to Nonlinear Systems
,”
AeroSense'97, Conference on Photonic Quantum Computing, Orlando FL
, Apr. 20–25,
Proc. SPIE
,
3068
, pp.
182
193
.
You do not currently have access to this content.