Abstract

Due to mobility impairment, a person might rely on wheelchairs, canes, and crutches for assistance but could face challenges when performing tasks such as grasping and manipulating objects due to limitations in reach and capability. To overcome these challenges, a multidegree-of-freedom robotic arm with an anthropomorphic robotic hand (ARH) could be used. In this research, we propose an architecture and then implement it toward the development of an assistive system to assist a person with object grasping. The architecture interlinks three functional modules to provide three operation modes to calibrate the system, train a user on how to execute a grasp, synthesize grasps, and execute a grasp. The developed system consists of a user input and feedback glove capable of capturing user inputs and providing grasp-related vibrotactile feedback, a coppeliasim virtual environment emulating the motions of the ARH, and an underactuated ARH capable of executing grasps while sensing grasp contact locations. The operation of the developed system is evaluated to determine the ability of a person to operate it and perform a grasp using two control methods; using a synthesized grasp or under real-time continuous control. The successful evaluation validates the architecture and the developed system to provide the ability to perform a grasp. The results of the evaluation provide confidence in expanding the system capabilities and using it to develop a database of grasp trajectories of objects with different geometries.

References

1.
Encarnação
,
P.
, and
Cook
,
A.
,
2017
,
Robotic Assistive Technologies: Principles and Practice
,
CRC Press
, Boca Raton, FL.
2.
Culbertson
,
H.
,
Schorr
,
S. B.
, and
Okamura
,
A. M.
,
2018
, “
Haptics: The Present and Future of Artificial Touch Sensation
,”
Annu. Rev. Control, Rob., Auton. Syst.
,
1
(
1
), pp.
385
409
.10.1146/annurev-control-060117-105043
3.
Sigrist
,
R.
,
Rauter
,
G.
,
Riener
,
R.
, and
Wolf
,
P.
,
2013
, “
Augmented Visual, Auditory, Haptic, and Multimodal Feedback in Motor Learning: A Review
,”
Psychon. Bull. Rev.
,
20
(
1
), pp.
21
53
.10.3758/s13423-012-0333-8
4.
Jones
,
L. A.
, and
Sarter
,
N. B.
,
2008
, “
Tactile Displays: Guidance for Their Design and Application
,”
Human Factors J. Human Factors Ergon. Soc.
,
50
(
1
), pp.
90
111
.10.1518/001872008X250638
5.
Hazra
,
S.
,
2018
, “
Inducing Vibro-Tactile Sensation at Mesoscale
,”
M.S. thesis
,
The University of Texas at Arlington
,
Arlington, TX
.http://hdl.handle.net/10106/29676
6.
Coppelia Robotics,
2020
, “
CoppeliaSim
,” Coppelia Robotics, Zürich, Switzerland, accessed Aug. 30, 2020, https://www.coppeliarobotics.com/
7.
Langevin
,
G.
,
2014
, “
Inmoov Hand
,” accessed Aug. 30, 2019, http://inmoov.fr/hand-and-forarm
8.
Lii
,
N. Y.
,
Chen
,
Z.
,
Pleintinger
,
B.
,
Borst
,
C. H.
,
Hirzinger
,
G.
, and
Schiele
,
A.
,
2010
, “
Toward Understanding the Effects of Visual- and Force-Feedback on Robotic Hand Grasping Performance for Space Teleoperation
,”
IEEE/RSJ International Conference on Intelligent Robots and Systems
,
Taipei, Taiwan
,
Oct. 18–22
, pp.
3745
3752
.10.1109/IROS.2010.5650186
9.
Fishel
,
J. A.
,
Oliver
,
T.
,
Eichermueller
,
M.
,
Barbieri
,
G.
,
Fowler
,
E.
,
Hartikainen
,
T.
,
Moss
,
L.
, and
Walker
,
R.
,
2020
, “
Tactile Telerobots for Dull, Dirty, Dangerous, and Inaccessible Tasks
,” IEEE International Conference on Robotics and Automation (
ICRA
),
Paris, France
,
May 31–Aug. 31
, pp.
11305
11310
.10.1109/ICRA40945.2020.9196888
10.
Handa
,
A.
,
Van Wyk
,
K.
,
Yang
,
W.
,
Liang
,
J.
,
Chao
,
Y.
,
Wan
,
Q.
,
Birchfield
,
S.
,
Ratliff
,
N.
, and
Fox
,
D.
,
2020
, “
DexPilot: Vision-Based Teleoperation of Dexterous Robotic Hand-Arm System
,” IEEE International Conference on Robotics and Automation (
ICRA
),
Paris, France
,
May 31–Aug. 31
, pp.
9164
9170
.10.1109/ICRA40945.2020.9197124
11.
Hu
,
J.
,
Watkins
,
D.
, and
Allen
,
P.
,
2023
, “
Teleoperated Robot Grasping in Virtual Reality Spaces
,” arXiv, p.
2301.13064
.10.48550/arXiv.2301.13064
12.
Li
,
S.
,
Rameshwar
,
R.
,
Votta
,
A. M.
, and
Onal
,
C. D.
,
2019
, “
Intuitive Control of a Robotic Arm and Hand System With Pneumatic Haptic Feedback
,”
IEEE Rob. Autom. Lett.
,
4
(
4
), pp.
4424
4430
.10.1109/LRA.2019.2937483
13.
Liarokapis
,
M. V.
,
Artemiadis
,
P. K.
, and
Kyriakopoulos
,
K. J.
,
2013
, “
Telemanipulation With the DLR/HIT II Robot Hand Using a Dataglove and a Low Cost Force Feedback Device
,”
21st Mediterranean Conference on Control and Automation
,
Platanias, Greece
,
June 25–28
, pp.
431
436
.10.1109/MED.2013.6608758
14.
Cerulo
,
I.
,
Ficuciello
,
F.
,
Lippiello
,
V.
, and
Siciliano
,
B.
,
2017
, “
Teleoperation of the SCHUNK S5FH Under-Actuated Anthropomorphic Hand Using Human Hand Motion Tracking
,”
Rob. Auton. Syst.
,
89
, pp.
75
84
.10.1016/j.robot.2016.12.004
15.
Chan
,
A.
,
MacLean
,
K.
, and
McGrenere
,
J.
,
2005
, “
Learning and Identifying Haptic Icons Under Workload
,”
First Joint Eurohaptics Conference and Symposium on Haptic Interfaces for Virtual Environment and Teleoperator Systems, World Haptics Conference
,
Pisa, Italy
,
Mar. 18–20
, pp.
432
439
.10.1109/WHC.2005.86
16.
Choi
,
H.
,
Crump
,
C.
,
Duriez
,
C.
,
Elmquist
,
A.
,
Hager
,
G.
,
Han
,
D.
,
Hearl
,
F.
,
Hodgins
,
J.
,
Jain
,
A.
,
Leve
,
F.
,
Li
,
C.
,
Meier
,
F.
,
Negrut
,
D.
,
Righetti
,
L.
,
Rodriguez
,
A.
,
Tan
,
J.
, and
Trinkle
,
J.
,
2021
, “
On the Use of Simulation in Robotics: Opportunities, Challenges, and Suggestions for Moving Forward
,”
Proc. Natl. Acad. Sci.
,
118
(
1
), p.
e1907856118
.10.1073/pnas.1907856118
17.
Lipton
,
J. I.
,
Fay
,
A. J.
, and
Rus
,
D.
,
2018
, “
Baxter's Homunculus: Virtual Reality Spaces for Teleoperation in Manufacturing
,”
IEEE Rob. Autom. Lett.
,
3
(
1
), pp.
179
186
.10.1109/LRA.2017.2737046
18.
Omarali
,
B.
,
Denoun
,
B.
,
Althoefer
,
K.
,
Jamone
,
L.
,
Valle
,
M.
, and
Farkhatdinov
,
I.
,
2020
, “
Virtual Reality Based Telerobotics Framework With Depth Cameras
,” 29th IEEE International Conference on Robot and Human Interactive Communication (
RO-MAN
),
Naples, Italy
,
Aug. 31–Sept. 4
, pp.
1217
1222
.10.1109/RO-MAN47096.2020.9223445
19.
Ozana
,
A.
,
Berman
,
S.
, and
Ganel
,
T.
,
2020
, “
Grasping Weber's Law in a Virtual Environment: The Effect of Haptic Feedback
,”
Front. Psychol.
,
11
, p.
57335
.10.3389/fpsyg.2020.573352
20.
Hazra
,
S.
,
Whitaker
,
S.
, and
Shiakolas
,
P. S.
,
2023
, “
Design and Implementation of a Behavioral Sequence Framework for Human-Robot Interaction Utilizing Brain-Computer Interface and Haptic Feedback
,”
J. Eng. Sci. Med. Diagn. Ther.
,
6
(
4
), p.
041003
.10.1115/1.4062341
21.
Abdul Rahaman
,
A. H.
,
2018
, “
On the Development of a Human Thumb Tracking Device for Telemanipulation
,”
M.S. thesis
,
The University of Texas at Arlington
,
Arlington, TX
.http://hdl.handle.net/10106/28296
22.
Smith, R.,
2006
, “
Open Dynamics Engine
,” accessed Dec. 2021, http://www.ode.org
23.
Coppelia Robotics,
2021
, “
CoppeliaSim User Manual
,” Coppelia Robotics, Zürich, Switzerland, accessed Sept. 20, 2021, https://www.coppeliarobotics.com/helpFiles/index.html
You do not currently have access to this content.