This paper proposes an approach for using force-controlled exploration data to update and register an a priori virtual fixture geometry to a corresponding deformed and displaced physical environment. An approach for safe exploration implementing hybrid motion/force control is presented on the slave robot side. During exploration, the shape and the local surface normals of the environment are estimated and saved in an exploration data set. The geometric data collected during this exploration scan are used to deform and register the a priori environment model to the exploration data set. The environment registration is achieved using a deformable registration based on the coherent point drift method. The task-description of the high-level assistive telemanipulation law, called a virtual fixture (VF), is then deformed and registered in the new environment. The new model is updated and used within a model-mediated telemanipulation framework. The approach is experimentally validated using a da-Vinci research kit (dVRK) master interface, a dVRK patient side manipulator, and a Cartesian stage robot. Experiments demonstrate that the updated VF and the updated model allow the users to improve their path following performance and to shorten their completion time when the updated path following VF is applied. The approach presented has direct bearing on a multitude of surgical applications including force-controlled ablation.

References

1.
Sotiras
,
A.
,
Davatzikos
,
C.
, and
Paragios
,
N.
,
2013
, “
Deformable Medical Image Registration: A Survey
,”
IEEE Trans. Med. Imaging
,
32
(
7
), pp.
1153
1190
.
2.
Mirota
,
D. J.
,
Ishii
,
M.
, and
Hager
,
G. D.
,
2011
, “
Vision-Based Navigation in Image-Guided Interventions
,”
Annu. Rev. Biomed. Eng.
,
13
(
1
), pp.
297
319
.
3.
Cash
,
D. M.
,
Sinha
,
T. K.
,
Chapman
,
W. C.
,
Terawaki
,
H.
,
Dawant
,
B. M.
,
Galloway
,
R. L.
, and
Miga
,
M. I.
,
2003
, “
Incorporation of a Laser Range Scanner Into Image-Guided Liver Surgery: Surface Acquisition, Registration, and Tracking
,”
Med. Phys.
,
30
(
7
), pp.
1671
1682
.
4.
Hayashibe
,
M.
,
2006
, “
Laser-Scan Endoscope System for Intraoperative Geometry Acquisition and Surgical Robot Safety Management
,”
Med. Image Anal.
,
10
(
4
), pp.
509
519
.
5.
Lathrop
,
R. A.
,
Rucker
,
D. C.
, and
Webster
,
R. J.
, III
,
2010
, “
Guidance of a Steerable Cannula Robot in Soft Tissue Using Preoperative Imaging and Conoscopic Surface Contour Sensing
,” International Conference on Robotics and Automation (
ICRA
), Anchorage, AK, May 3–7, pp.
5601
5606
.
6.
Xia
,
T.
,
Leonard
,
S.
,
Kandaswamy
,
I.
,
Blank
,
A.
,
Whitcomb
,
L. L.
, and
Kazanzides
,
P.
,
2013
, “
Model-Based Telerobotic Control With Virtual Fixtures for Satellite Servicing Tasks
,”
IEEE International Conference on Robotics and Automation
(
ICRA
), Karlsruhe, Germany, May 6–10, pp.
1479
1484
.
7.
Mitra
,
P.
, and
Niemeyer
,
G.
,
2008
, “
Model-Mediated Telemanipulation
,”
Int. J. Rob. Res.
,
27
(
2
), pp.
253
262
.
8.
Kazanzides
,
P.
,
Chen
,
Z.
,
Deguet
,
A.
,
Fischer
,
G. S.
,
Taylor
,
R. H.
, and
DiMaio
,
S.
,
2014
, “
An Open-Source Research Kit for the da Vinci[Textregistered] Surgical Robot
,”
IEEE International Conference on Robotics and Automation
(
ICRA
), Hong Kong, China, May 31–June 7, pp. 6434–6439.
9.
Galea
,
A.
, and
Howe
,
R.
,
2002
, “
Tissue Stiffness From Tactile Imaging
,”
Second Joint 24th Annual Conference and the Annual Fall Meeting of the Biomedical Engineering Society Conference in Engineering in Medicine and Biology
(
EMBS/BMES
), Houston, TX, Oct. 23–26, Vol.
2
, pp.
935
936
.
10.
Kesner
,
S. B.
, and
Howe
,
R. D.
,
2011
, “
Discriminating Tissue Stiffness With a Haptic Catheter: Feeling the Inside of the Beating Heart
,”
World Haptics Conference
(
WHC
), Istanbul, Turkey, June 21–24, pp.
13
18
.
11.
Egorov
,
V.
,
Van Raalte
,
H.
, and
Sarvazyan
,
A. P.
,
2010
, “
Vaginal Tactile Imaging
,”
IEEE Trans. Biomed. Eng.
,
57
(
7
), pp.
1736
1744
.
12.
Liu
,
H.
,
Noonan
,
D. P.
,
Challacombe
,
B. J.
,
Dasgupta
,
P.
,
Seneviratne
,
L. D.
, and
Althoefer
,
K.
,
2010
, “
Rolling Mechanical Imaging for Tissue Abnormality Localization During Minimally Invasive Surgery
,”
IEEE Trans. Biomed. Eng.
,
57
(
2
), pp.
404
414
.
13.
Xu
,
K.
, and
Simaan
,
N.
,
2010
, “
Intrinsic Wrench Estimation and Its Performance Index for Multisegment Continuum Robots
,”
IEEE Trans. Rob.
,
26
(
3
), pp.
555
561
.
14.
Nichols
,
K.
, and
Okamura
,
A. M.
,
2013
, “
Autonomous Robotic Palpation: Machine Learning Techniques to Identify Hard Inclusions in Soft Tissues
,”
IEEE International Conference on Robotics and Automation
(
ICRA
), Karlsruhe, Germany, May 6–10, pp.
4384
4389
.
15.
Bajo
,
A.
, and
Simaan
,
N.
,
2016
, “
Hybrid Motion/Force Control of Multi-Backbone Continuum Robots
,”
Int. J. Rob. Res.
,
35
(
4
), pp.
422
434
.
16.
Goldman
,
R. E.
,
Bajo
,
A.
, and
Simaan
,
N.
,
2013
, “
Algorithms for Autonomous Exploration and Estimation in Compliant Environments
,”
Robotica
,
31
(
1
), pp.
71
88
.
17.
Sanan
,
S.
,
Tully
,
S.
,
Bajo
,
A.
,
Simaan
,
N.
, and
Choset
,
H.
,
2014
, “
Simultaneous Compliance and Registration Estimation for Robotic Surgery
,”
Robotics: Science and Systems
, Berkeley, CA.
18.
Srivatsan
,
R. A.
,
Ayvali
,
E.
,
Wang
,
L.
,
Roy
,
R.
,
Simaan
,
N.
, and
Choset
,
H.
,
2016
, “
Complementary Model Update: A Method for Simultaneous Registration and Stiffness Mapping in Flexible Environments
,”
IEEE International Conference on Robotics and Automation
(
ICRA
), Stockholm, Sweden, May 16–21, pp. 924–930.
19.
Chalasani
,
P.
,
Wang
,
L.
,
Roy
,
R.
,
Simaan
,
N.
,
Taylor
,
R. H.
, and
Kobilarov
,
M.
,
2016
, “
Concurrent Nonparametric Estimation of Organ Geometry and Tissue Stiffness Using Continuous Adaptive Palpation
,”
IEEE International Conference on Robotics and Automation
(
ICRA
), Stockholm, Sweden, May 16–21, pp. 4164–4171.
20.
Ayvali
,
E.
,
Srivatsan
,
R. A.
,
Wang
,
L.
,
Roy
,
R.
,
Simaan
,
N.
, and
Choset
,
H.
,
2016
, “
Using Bayesian Optimization to Guide Probing of a Flexible Environment for Simultaneous Registration and Stiffness Mapping
,”
IEEE International Conference on Robotics and Automation
(
ICRA
), Stockholm, Sweden, May 16–21, pp. 931–936.
21.
Myronenko
,
A.
, and
Song
,
X.
,
2010
, “
Point Set Registration: Coherent Point Drift
,”
IEEE Trans. Pattern Anal. Mach. Intell.
,
32
(
12
), pp.
2262
2275
.
22.
Billings
,
S. D.
,
Boctor
,
E. M.
, and
Taylor
,
R. H.
,
2015
, “
Iterative Most-Likely Point Registration (Imlp): A Robust Algorithm for Computing Optimal Shape Alignment
,”
PloS One
,
10
(
3
), p. e0117688.
23.
Wang
,
L.
,
Chen
,
Z.
,
Chalasani
,
P.
,
Pile
,
J.
,
Kazanzides
,
P.
,
Taylor
,
R. H.
, and
Simaan
,
N.
,
2016
, “
Updating Virtual Fixture From Exploration Data in Force-Controlled Model-Based Telemanipulation
,”
ASME
Paper No. DETC2016-59305.
24.
Jung
,
M. Y.
,
Deguet
,
A.
, and
Kazanzides
,
P.
,
2010
, “
A Component-Based Architecture for Flexible Integration of Robotic Systems
,”
IEEE/RSJ International Conference on Intelligent Robots and Systems
(
IROS
), Taipei, Taiwan, Oct. 18–22, pp.
6107
6112
.
25.
Chen
,
Z.
,
Deguet
,
A.
,
Taylor
,
R.
,
DiMaio
,
S.
,
Fischer
,
G.
, and
Kazanzides
,
P.
,
2013
, “
An Open-Source Hardware and Software Platform for Telesurgical Robot Research
,” Department of Computer Science, Johns Hopkins University, Baltimore, MD, pp. 22–26.
26.
Khatib
,
O.
,
1987
, “
A Unified Approach for Motion and Force Control of Robot Manipulators: The Operational Space Formulation
,”
IEEE J. Rob. Autom.
,
RA-3
(
1
), pp.
43
53
.
27.
Featherstone
,
R.
,
Thiebaut
,
S.
, and
Khatib
,
O.
,
1999
, “
A General Contact Model for Dynamically-Decoupled Force/Motion Control
,”
IEEE International Conference on Robotics and Automation
(
ICRA
), Detroit, MI, May 10–15, pp.
3281
3286
.
28.
Whitney
,
D. E.
,
1969
, “
Resolved Motion Rate Control of Manipulators and Human Prostheses
,”
IEEE Trans. Man-Mach. Syst.
,
10
(
2
), pp.
47
53
.
29.
Kapoor
,
A.
,
Li
,
M.
, and
Taylor
,
R.
,
2006
, “
Constrained Control for Surgical Assistant Robots
,”
IEEE International Conference on Robotics and Automation
(
ICRA
), Orlando, FL, May 15–19, pp.
231
236
.
30.
Yoshikawa
,
T.
, and
Sudou
,
A.
,
1993
, “
Dynamic Hybrid Position/Force Control of Robot Manipulators—On-Line Estimation of Unknown Constraint
,”
IEEE Trans. Rob. Autom.
,
9
(
2
), pp.
220
226
.
31.
Namvar
,
M.
, and
Aghili
,
F.
,
2005
, “
Adaptive Force-Motion Control of Coordinated Robots Interacting With Geometrically Unknown Environments
,”
IEEE Trans. Rob.
,
21
(
4
), pp.
678
694
.
32.
Yin
,
Y.
,
Hu
,
H.
, and
Xia
,
Y.
,
2004
, “
Active Tracking of Unknown Surface Using Force Sensing and Control Technique for Robot
,”
Sens. Actuators A
,
112
(
2
), pp.
313
319
.
33.
Karayiannidis
,
Y.
, and
Doulgeri
,
Z.
,
2009
, “
Adaptive Control of Robot Contact Tasks With On-Line Learning of Planar Surfaces
,”
Automatica
,
45
(
10
), pp.
2374
2382
.
34.
Kwartowitz
,
D. M.
,
Herrell
,
S. D.
, and
Galloway
,
R. L.
,
2006
, “
Toward Image-Guided Robotic Surgery: Determining Intrinsic Accuracy of the da Vinci Robot
,”
Int. J. Comput. Assisted Radiol. Surg.
,
1
(
3
), pp.
157
165
.
35.
Linte
,
C. A.
,
Moore
,
J.
, and
Peters
,
T. M.
,
2010
, “
How Accurate is Accurate Enough? A Brief Overview on Accuracy Considerations in Image-Guided Cardiac Interventions
,”
Annual International Conference of the IEEE Engineering in Medicine and Biology Society
(
EMBC
), Buenos Aires, Argentina, Aug. 3–Sept. 4, pp.
2313
2316
.
36.
Krücker
,
J.
,
Xu
,
S.
,
Glossop
,
N.
,
Viswanathan
,
A.
,
Borgert
,
J.
,
Schulz
,
H.
, and
Wood
,
B. J.
,
2007
, “
Electromagnetic Tracking for Thermal Ablation and Biopsy Guidance: Clinical Evaluation of Spatial Accuracy
,”
J. Vasc. Interventional Radiol.
,
18
(
9
), pp.
1141
1150
.
You do not currently have access to this content.