We present a multiple Kinects based exteroceptive sensing framework to achieve safe human-robot collaboration during assembly tasks. Our approach is mainly based on a real-time replication of the human and robot movements inside a physics-based simulation of the work cell. This enables the evaluation of the human-robot separation in a 3D Euclidean space, which can be used to generate safe motion goals for the robot. For this purpose, we develop an N-Kinect system to build an explicit model of the human and a roll-out strategy, in which we forward-simulate the robot's trajectory into the near future. Now, we use a precollision strategy that allows a human to operate in close proximity with the robot, while pausing the robot's motion whenever an imminent collision between the human model and any part of the robot is detected. Whereas most previous range based methods analyzed the physical separation based on depth data pertaining to 2D projections of robot and human, our approach evaluates the separation in a 3D space based on an explicit human model and a forward physical simulation of the robot. Real-time behavior (≈ 30 Hz) observed during experiments with a 5 DOF articulated robot and a human safely collaborating to perform an assembly task validate our approach.

References

References
1.
Bicchi
,
A.
, and
Tonietti
,
G.
,
2004
, “
Fast and Soft Arm Tactics: Dealing With the Safety-Performance Trade-Off in Robot Arms Design and Control
,”
IEEE Rob. Autom. Mag.
,
11
(
2
), pp.
22
33
.10.1109/MRA.2004.1310939
2.
Haddadin
,
S.
,
Albu-Schaffer
,
A.
, and
Hirzinger
,
G.
,
2010
, “
Safety Analysis for a Human-Friendly Manipulator
,”
Int. J. Soc. Robot.
,
2
, pp.
235
252
.10.1007/s12369-010-0053-z
3.
Haddadin
,
S.
,
Albu-Schaffer
,
A.
, and
Hirzinger
,
G.
,
2009
, “
Requirements for Safe Robots: Measurements, Analysis and New Insights
,”
Int. J. Robot. Res.
,
28
(
11–12
), pp.
1507
1527
.10.1177/0278364909343970
4.
Heinzmann
,
J.
, and
Zelinsky
,
A.
,
2003
, “
Quantitative Safety Guarantees for Physical Human-Robot Interaction
,”
Int. J. Robot. Res.
,
22
(
7–8
), pp.
479
504
.10.1177/02783649030227004
5.
Vogel
,
C.
,
Poggendorf
,
M.
,
Walter
,
C.
, and
Elkmann
,
N.
,
2011
, “
Towards Safe Physical Human-Robot Collaboration: A Projection-Based Safety System
,”
Intelligent Robots and Systems (IROS), 2011 IEEE/RSJ International Conference on
, pp.
3355
3360
.
6.
Balan
,
L.
, and
Bone
,
G. M.
,
2006
, “
Real-Time 3d Collision Avoidance Method for Safe Human and Robot Coexistence
,”
Intelligent Robots and Systems, 2006 IEEE/RSJ International Conference on
, pp.
276
282
.
7.
Flacco
,
F.
, and
De Luca
,
A.
,
2010
, “
Multiple Depth/Presence Sensors: Integration and Optimal Placement for Human/Robot Coexistence
,”
Robotics and Automation (ICRA), 2010 IEEE International Conference on
, pp.
3916
3923
.
8.
Flacco
,
F.
,
Kroger
,
T.
,
De Luca
,
A.
, and
Khatib
,
O.
,
2012
, “
A Depth Space Approach to Human-Robot Collision Avoidance
,”
Robotics and Automation (ICRA), 2012 IEEE International Conference on
, pp.
338
345
.
9.
Kuhn
,
S.
,
Gecks
,
T.
, and
Henrich
,
D.
, 2006, “
Velocity Control for Safe Robot Guidance Based on Fused Vision and Force/Torque Data
,”
IEEE International Conference on Multisensor Fusion and Integration for Intelligent Systems
, pp.
485
492
.
10.
Kulic
,
D.
, and
Croft
,
E.
,
2007
, “
Pre-Collision Safety Strategies for Human-Robot Interaction
,”
Auton. Rob.
,
22
(2), pp.
149
164
.10.1007/s10514-006-9009-4
11.
Najmaei
,
N.
, and
Kermani
,
M.
,
2010
, “
Prediction-Based Reactive Control Strategy for Human-Robot Interactions
,”
Robotics and Automation (ICRA), 2010 IEEE International Conference on
, pp.
3434
3439
.
12.
Schiavi
,
R.
,
Bicchi
,
A.
, and
Flacco
,
F.
,
2009
, “
Integration of Active and Passive Compliance Control for Safe Human-Robot Coexistence
,”
Robotics and Automation, 2009. ICRA’09. IEEE International Conference on
, pp.
259
264
.
13.
Haddadin
,
S.
,
Albu-Schaffer
,
A.
,
De Luca
,
A.
, and
Hirzinger
,
G.
,
2008
, “
Collision Detection and Reaction: A Contribution to Safe Physical Human-Robot Interaction
,”
Intelligent Robots and Systems, 2008. IROS 2008. IEEE/RSJ International Conference on
, pp.
3356
3363
.
14.
Shin
,
D.
,
Sardellitti
,
I.
,
Park
,
Y.-L.
,
Khatib
,
O.
, and
Cutkosky
,
M. R.
,
2010
, “
Design and Control of a Bio-Inspired Human-Friendly Robot
,”
Int. J. Robot. Res.
,
29
(5), pp.
571
584
.
15.
Marvel
,
J. A.
,
2013
, “
Performance Metrics of Speed and Separation Monitoring in Shared Workspaces
,”
IEEE. Trans. Autom. Sci. Eng.
,
10
(
2
), pp.
405
414
.10.1109/TASE.2013.2237904
16.
De Santis
,
A.
,
Siciliano
,
B.
,
De Luca
,
A.
, and
Bicchi
,
A.
,
2008
. “
An Atlas of Physical Human–Robot Interaction
,”
Mech. Mach. Theory
,
43
(
3
), pp.
253
270
.10.1016/j.mechmachtheory.2007.03.003
17.
Laborie
,
P.
, and
Ghallab
,
M.
,
1995
, “
Planning With Sharable Resource Constraints
,”
International Joint Conference on Artificial Intelligence, LAWRENCE ERLBAUM ASSOCIATES LTD
, Vol.
14
, pp.
1643
1651
.
18.
Gupta
,
S. K.
,
Paredis
,
C.
,
Sinha
,
R.
, and
Brown
,
P.
,
2001
, “
Intelligent Assembly Modeling and Simulation
,”
Assem. Autom.
,
21
(
3
), pp.
215
235
.10.1108/01445150110399151
19.
Morato
,
C.
,
Kaipa
,
K. N.
, and
Gupta
,
S. K.
,
2013
, “
Improving Assembly Precedence Constraint Generation by Utilizing Motion Planning and Part Interaction Clusters
,”
Comput.-Aided Des.
,
45
(
11
), pp.
1349
1364
.10.1016/j.cad.2013.06.005
20.
Brough
,
J.
,
Schwartz
,
M.
,
Gupta
,
S. K.
,
Anand
,
D.
,
Kavetsky
,
R.
, and
Pettersen
,
R.
,
2007
, “
Towards Development of a Virtual Environment-Based Training System for Mechanical Assembly Operations
,”
Virtual Reality
,
11
(
4
), pp.
189
206
.10.1007/s10055-007-0076-4
21.
Gupta
,
S. K.
,
Anand
,
D.
,
Brough
,
J.
,
Kavetsky
,
R.
,
Schwartz
,
M.
, and
Thakur
,
A.
,
2008
, “
A Survey of the Virtual Environments-Based Assembly Training Applications
,”
Virtual Manufacturing Workshop
,
Turin, Italy
, October.
22.
Kaipa
,
K. N.
,
Morato
,
C.
,
Zhao
,
B.
, and
Gupta
,
S. K.
,
2012
, “
Instruction Generation for Assembly Operation Performed by Humans
,”
ASME Computers and Information in Engineering Conference
,
Chicago, IL
, August 2–15, 2012.
23.
Smith
,
C.
, and
Christensen
,
H. I.
,
2009
, “
Wiimote Robot Control Using Human Motion Models
,”
Intelligent Robots and Systems, 2009. IROS 2009. IEEE/RSJ International Conference on, IEEE
, pp.
5509
5515
.
24.
Morato
,
C.
,
Kaipa
,
K. N.
,
Zhao
,
B.
, and
Gupta
,
S. K.
,
2013
, “
Safe Human Robot Interaction by using Exteroceptive Sensing Based Human Modeling
,”
ASME Computers and Information in Engineering Conference
,
Portland, OR
, August.
25.
Henrich
,
D.
, and
Gecks
,
T.
,
2008
, “
Multi-Camera Collision Detection Between Known and Unknown Objects
,”
Distributed Smart Cameras, 2008. ICDSC 2008. Second ACM/IEEE International Conference on
, pp.
1
10
.
26.
Fischer
,
J.
,
H. B.
, and
Schilling
,
A.
,
2007
, “
Using Time-of-Flight Range Data for Occlusion Handling in Augmented Reality
,”
Eurographics Symposium on Virtual Environments (EGVE)
.
27.
Valentini
,
P. P.
,
2012
, “
Natural Interface in Augmented Reality Interactive Simulations
,”
Virtual Phys. Prototyping
,
7
, pp.
137
151
.10.1080/17452759.2012.682332
28.
Najmaei
,
N.
,
Kermani
,
M.
, and
Al-Lawati
,
M.
,
2011
, “
A New Sensory System for Modeling and Tracking Humans Within Industrial Work Cells
,”
IEEE Trans. Instrum. Meas.
,
60
(
4
), pp.
1227
1236
.10.1109/TIM.2010.2083350
29.
Maybeck
,
P.
,
1979
,
Stochastic Models, Estimation, and Control
,
Academic Press, Inc
,
New York
, Vol.
1
.
30.
Corrales
,
J. A.
,
Candelas
,
F. A.
, and
Torres
,
F.
,
2008
, “
Hybrid Tracking of Human Operators Using imu/uwb Data Fusion by a Kalman Filter
,”
Proceedings of the 3rd ACM/IEEE International Conference on Human Robot Interaction, HRI’08, ACM
, pp.
193
200
.
31.
Andrieu
,
C.
, and
Doucet
,
A.
,
2002
, “
Particle Filtering for Partially Observed Gaussian State Space Models
,”
J. R. Stat. Soc.
,
64
, pp.
827
836
.10.1111/1467-9868.00363
32.
Davies
,
S.
,
2007
, “
Watching Out for the Workers [Safety Workstations]
,”
Manuf., IET.
,
86
(
4
), pp.
32
34
.10.1049/me:20070409
33.
ISO 10218-1:2011, “Robots and Robotic Devices—Safety Requirements for Industrial Robots—Part 1: Robots,” International Organization for Standardization, Geneva, Switzerland, 2011. http://www.iso.org/iso/home/store/catalogue_ics/catalogue_detail_ics.htm?csnumber=51330
34.
ISO 10218-2:2011, “Robots and Robotic Devices—Safety Requirements for Industrial Robots—Part 2: Industrial Robot Systems and Integration,” International Organization for Standardization, Geneva, Switzerland, 2011. http://www.iso.org/iso/catalogue_detail.htm?csnumber=41571.
You do not currently have access to this content.