The ability to track human operators' hand usage when working in production plants and factories is critically important for developing realistic digital factory simulators as well as manufacturing process control. We propose a proof-of-concept instrumented glove with only a few strain gage sensors and a microcontroller that continuously tracks and records the hand configuration during actual use. At the heart of our approach is a trainable system that can predict the fourteen joint angles in the hand using only a small set of strain sensors. First, ten strain gages are placed at various joints in the hand to optimize the sensor layout using the English letters in the American Sign Language (ASL) as a benchmark for assessment. Next, the best sensor configurations for three through ten strain gages are computed using a support vector machine (SVM) classifier. Following the layout optimization, our approach learns a mapping between the sensor readouts to the actual joint angles optically captured using a Leap Motion system. Five regression methods including linear, quadratic, and neural regression are then used to train the mapping between the strain gage data and the corresponding joint angles. The final proposed model involves four strain gages mapped to the fourteen joint angles using a two-layer feed-forward neural network (NN).

References

References
1.
Wentai
,
Z.
,
Yu
,
J. Z.
,
Zhu
,
F.
,
Zhu
,
Y.
,
Gecer Ulu
,
N.
,
Arisoy
,
B.
, and
Kara
,
L. B.
,
2018
, “
High Degree of Freedom Hand Pose Tracking Using Limited Strain Sensing and Optical Training
,”
ASME
Paper No. DETC2018-85870http://proceedings.asmedigitalcollection.asme.org/proceeding.aspx?articleid=2713114.
2.
Kramer
,
J. F.
,
George
,
W. R.
, and
Lindener
,
P.
,
1994
, “
Strain-Sensing Goniometers, Systems and Recognition Algorithms
,” U.S. Patent 5,280,265.
3.
Lorussi
,
F.
,
Scilingo
,
E. P.
,
Tesconi
,
M.
,
Tognetti
,
A.
, and
De Rossi
,
D.
,
2005
, “
Strain Sensing Fabric for Hand Posture and Gesture Monitoring
,”
IEEE Trans. Inf. Technol. Biomed.
,
9
(
3
), pp.
372
381
.
4.
Jing
,
L.
,
Zhou
,
Y.
,
Cheng
,
Z.
, and
Wang
,
J.
,
2011
, “
A Recognition Method for One-Stroke Finger Gestures Using a MEMS 3D Accelerometer
,”
IEICE Trans.
,
94D
(
5
), pp.
1062
1072
.
5.
Rendl
,
C.
,
Kim
,
D.
,
Fanello
,
S. R.
,
Parzer
,
P.
,
Rhemann
,
C.
,
Taylor
,
J.
,
Zirkl
,
M.
,
Scheipl
,
G.
,
Rothländer
,
T.
,
Haller
,
M. J.
, and
Izadi
,
S.
,
2014
, “
FlexSense: A Transparent Self-Sensing Deformable Surface
,” 27th Annual ACM Symposium on User Interface Software and Technology (UIST‘14), Honolulu, HI, Oct. 5–8.
6.
Carbonaro
,
N.
,
Mura
,
G. D.
,
Lorussi
,
F.
,
Paradiso
,
R.
,
De Rossi
,
D.
, and
Tognetti
,
A.
,
2014
, “
Exploiting Wearable Goniometer Technology for Motion Sensing Gloves
,”
IEEE J. Biomed. Health Inf.
,
18
(
6
), pp.
1788
1795
.
7.
Hammond
,
F. L.
,
Mengüç
,
Y.
, and
Wood
,
R. J.
,
2014
, “
Toward a Modular Soft Sensor-Embedded Glove for Human Hand Motion and Tactile Pressure Measurement
,”
IEEE/RSJ International Conference on Intelligent Robots and Systems
(
IROS
), Chicago, IL, Sept. 14–18, pp.
4000
4007
.http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.726.100&rep=rep1&type=pdf
8.
Chen
,
F.-S.
,
Fu
,
C.-M.
, and
Huang
,
C.-L.
,
2003
, “
Hand Gesture Recognition Using a Real-Time Tracking Method and Hidden Markov Models
,”
Image Vision Comput.
,
21
(
8
), pp.
745
758
.
9.
Bretzner
,
L.
,
Laptev
,
I.
, and
Lindeberg
,
T.
,
2002
, “
Hand Gesture Recognition Using Multi-Scale Colour Features, Hierarchical Models and Particle Filtering
,”
Fifth IEEE International Conference on Automatic Face and Gesture Recognition
(
FGR 2002
), Washington, DC, May 20–21, pp.
423
428
.http://www.irisa.fr/vista/Papers/2002_fgr_bretzner.pdf
10.
Ren
,
Z.
,
Meng
,
J.
,
Yuan
,
J.
, and
Zhang
,
Z.
,
2011
, “
Robust Hand Gesture Recognition With Kinect Sensor
,”
19th ACM International Conference on Multimedia
(
ACM
), Scottsdale, AZ, Nov. 28–Dec. 1, pp.
759
760
.http://web.cs.ucla.edu/~zhou.ren/Ren_Meng_Yuan_Zhang_MM11.pdf
11.
Biswas
,
K. K.
, and
Basu
,
S. K.
,
2011
, “
Gesture Recognition Using Microsoft Kinect®
,”
Fifth International Conference on Automation, Robotics and Applications (ICARA)
, Wellington, New Zealand, Dec. 6–8, pp.
100
103
.
12.
Marin
,
G.
,
Dominio
,
F.
, and
Zanuttigh
,
P.
,
2014
, “
Hand Gesture Recognition With Leap Motion and Kinect Devices
,”
IEEE International Conference on Image Processing
(
ICIP
), Paris, France, Oct. 27–30, pp.
1565
1569
.
13.
Wang
,
R. Y.
, and
Popović
,
J.
,
2009
, “
Real-Time Hand-Tracking With a Color Glove
,”
ACM SIGGRAPH 2009 Papers (SIGGRAPH '09)
, New Orleans, LO, Aug. 3–7.
14.
da Silva
,
A. F.
,
Goncalves
,
A. F.
,
Mendes
,
P. M.
, and
Correia
,
J. H.
,
2011
, “
FBG Sensing Glove for Monitoring Hand Posture
,”
Sens. J. IEEE
,
11
(
10
), pp.
2442
2448
.
15.
Yuan
,
S.
,
Ye
,
Q.
,
Stenger
,
B.
,
Jain
,
S.
, and
Kim
,
T.
,
2017
, “
BigHand2.2M Benchmark: Hand Pose Dataset and State of the Art Analysis
,”
IEEE Conference on Computer Vision and Pattern Recognition (CVPR)
,
Honolulu, HI
, pp.
2605
2613
.
16.
Andrea
,
T.
,
Matthias
,
S.
,
Anastasia
,
T.
,
Sofien
,
B.
,
Mario
,
B.
, and
Mark
,
P.
,
2015
, “
Robust Articulated-ICP for Real-Time Hand Tracking
,”
Computer Graphics Forum (Proceedings of SGP)
,
34
(
5
), pp.
101
114
.
17.
Zappi
,
P.
,
Stiefmeier
,
T.
,
Farella
,
E.
,
Roggen
,
D.
,
Benini
,
L.
, and
Troster
,
G.
,
2007
, “
Activity Recognition From on-Body Sensors by Classifier Fusion: Sensor Scalability and Robustness
,”
Third International Conference on Intelligent Sensors, Sensor Networks and Information
(ISSNIP), Melbourne, Australia, Dec. 3–6, pp.
281
286
.
18.
Lee
,
S.-W.
, and
Mase
,
K.
,
2002
, “
Activity and Location Recognition Using Wearable Sensors
,”
IEEE Pervasive Comput.
,
1
(
3
), pp.
24
32
.
19.
Mattmann
,
C.
,
Amft
,
O.
,
Harms
,
H.
,
Troster
,
G.
, and
Clemens
,
F.
,
2007
, “
Recognizing Upper Body Postures Using Textile Strain Sensors
,”
11th IEEE International Symposium on Wearable Computers
(ISWC 2007), Boston, MA, Oct. 11–13, pp.
29
36
.
20.
Kern
,
N.
,
Schiele
,
B.
, and
Schmidt
,
A.
,
2003
, “
Multi-Sensor Activity Context Detection for Wearable Computing
,” European Symposium on Ambient Intelligence, Springer, Berlin, pp.
220
232
.
21.
ProGlove,
2015
, “
1st Smart Glove for Industries
,” ProGlove, Munich, Germany, accessed Mar. 2, 2017, http://www.proglove.de
22.
Myo,
2013
, “
Myo Gesture Control Armband
,” North Inc., Brooklyn, NY, accessed Mar. 2, 2017, https://www.myo.com
23.
Cyber Glove Systems
,
2016
, “
MoCap Glove System
,” CyberGlove Systems LLC, San Jose, CA, accessed Mar. 11, 2018, http://www.cyberglovesystems.com
24.
VR Guide
,
2017
, “
Samsung Rink: Buy VR Guide
,” Andreas Johannsen, Washington, DC, accessed Mar. 2, 2017, http://www.buyvrguide.com/vr-controllers/samsung-rink/
You do not currently have access to this content.