Abstract

Accurate registration of Cartesian coordinate systems is necessary to facilitate metrology-based solutions for industrial robots in production environments. Conducting coordinate registration between industrial robots and their metrological systems requires measuring multiple points in the robot’s and sensor system’s coordinate frames. However, operators lack intuitive tools to interface, visualize, and characterize the quality of the selected points in the robot workspace for robot–sensor coordinate registration. This article proposes an augmented reality system for human-in-the-loop, robot–sensor coordinate registration to efficiently record and visualize the pose-dependent quality of computing the robot–sensor transformation. Furthermore, this work establishes metrics to define the relative quality of measurement points used in robot–sensor coordinate registration, which is shown by the augmented reality application. Experiments were conducted demonstrating the augmented reality environment in addition to investigating the pose dependency of the measurement point quality. The results indicate that the proposed metrics highlight the dependency of the poses on both robot and sensor placement and that the augmented reality system can provide a human-in-the-loop interface for robot–sensor coordinate registration.

References

1.
Jiang
,
Y.
,
Yu
,
L.
,
Jia
,
H.
,
Zhao
,
H.
, and
Xia
,
H.
,
2020
, “
Absolute Positioning Accuracy Improvement in an Industrial Robot
,”
Sensors
,
20
(
16
), p.
4354
.
2.
Ji
,
W.
, and
Wang
,
L.
,
2019
, “
Industrial Robotic Machining: A Review
,”
Int. J. Adv. Manuf. Technol.
,
103
, pp.
1239
1255
.
3.
Marvel
,
J. A.
,
Bostelman
,
R.
, and
Falco
,
J.
,
2018
, “
Multi-Robot Assembly Strategies and Metrics
,”
ACM Comput. Surv.
,
51
(
1
), pp.
1
32
.
4.
Marvel
,
J. A.
,
Messina
,
E.
,
Antonishek
,
B.
,
van Wyk
,
K.
, and
Fronczek
,
L. J.
,
2015
, “
Tools for Robotics in SME Workcells: Challenges and Approaches for Calibration and Registration
,”
NIST Internal Report
.
5.
Nguyen
,
V.
,
Cvitanic
,
T.
,
Baxter
,
M.
,
Ahlin
,
K.
,
Johnson
,
J.
,
Freeman
,
P.
,
Balakirsky
,
S.
,
Brown
,
A.
, and
Melkote
,
S.
,
2021
, “
Precision Robotic Milling of Fiberglass Shims in Aircraft Wing Assembly Using Laser Tracker Feedback
,”
SAE Int. J. Aerosp.
,
15
(
1
), pp.
87
97
.
6.
Wasielewski
,
S.
, and
Strauss
,
O.
,
1995
, “
Calibration of a Multi-Sensor System Laser Rangefinder/Camera
,”
Proceedings of the Intelligent Vehicles’ 95. Symposium
,
Detroit, MI
,
Sept. 25–26
, IEEE, pp.
472
477
.
7.
Hu
,
J.
, and
Pagilla
,
P. R.
,
2021
, “
Dual-Edge Robotic Gear Chamfering With Registration Error Compensation
,”
Rob. Comput. Integr. Manuf.
,
69
, p.
102082
.
8.
Chen
,
Y. Q.
,
Huang
,
J.
,
Xu
,
H. P.
, and
Ma
,
Z.
,
2010
, “
Calibration and Registration for Inspection Robot With Contact Probe
,”
Adv. Mater. Res.
,
97–101
, pp.
4319
4323
.
9.
Motta
,
J. M. S. T.
,
de Carvalho
,
G. C.
, and
McMaster
,
R. S.
,
2001
, “
Robot Calibration Using a 3D Vision-Based Measurement System With a Single Camera
,”
Rob. Comput. Integr. Manuf.
,
17
(
6
), pp.
487
497
.
10.
Özgüner
,
O.
,
Shkurti
,
T.
,
Huang
,
S.
,
Hao
,
R.
,
Jackson
,
R. C.
,
Newman
,
W. S.
, and
Çavuşoğlu
,
M. C.
,
2020
, “
Camera-Robot Calibration for the da Vinci Robotic Surgery System
,”
IEEE Trans. Autom. Sci. Eng.
,
17
(
4
), pp.
2154
2161
.
11.
Kotake
,
D.
,
Uchiyama
,
S.
, and
Yamamoto
,
H.
,
2004
, “
A Marker Calibration Method Utilizing a Priori Knowledge on Marker Arrangement
,”
Third IEEE and ACM International Symposium on Mixed and Augmented Reality
,
Arlington, VA
,
Nov. 5
, pp.
89
98
.
12.
Cvitanic
,
T.
,
Nguyen
,
V.
, and
Melkote
,
S. N.
,
2020
, “
Pose Optimization in Robotic Machining Using Static and Dynamic Stiffness Models
,”
Rob. Comput. Integr. Manuf.
,
66
, p.
101992
.
13.
van Wyk
,
K.
, and
Marvel
,
J. A.
,
2017
, “
Strategies for Improving and Evaluating Robot Registration Performance
,”
IEEE Trans. Autom. Sci. Eng.
,
15
(
1
), pp.
320
328
.
14.
Cen
,
L.
, and
Melkote
,
S. N.
,
2017
, “
CCT-Based Mode Coupling Chatter Avoidance in Robotic Milling
,”
J. Manuf. Processes
,
29
, pp.
50
61
.
15.
Azuma
,
R. T.
,
1997
, “
A Survey of Augmented Reality
,”
Pres. Teleop. Virt. Environ.
,
6
(
4
), pp.
355
385
.
16.
Li
,
C.
,
Zheng
,
P.
,
Li
,
S.
,
Pang
,
Y.
, and
Lee
,
C. K. M.
,
2022
, “
AR-Assisted Digital Twin-Enabled Robot Collaborative Manufacturing System With Human-in-the-Loop
,”
Rob. Comput. Integr. Manuf.
,
76
, p.
102321
.
17.
Eiriksson
,
E. R.
,
Pedersen
,
D. B.
,
Frisvad
,
J. R.
,
Skovmand
,
L.
,
Heun
,
V.
,
Maes
,
P.
, and
Aanæs
,
H.
,
2017
, “
Augmented Reality Interfaces for Additive Manufacturing
,”
Scandinavian Conference on Image Analysis
,
Tromsø, Norway
,
June 12–14
, Springer, pp.
515
525
.
18.
Fuchs
,
H.
,
Livingston
,
M. A.
,
Raskar
,
R.
,
Keller
,
K.
,
Crawford
,
J. R.
,
Rademacher
,
P.
,
Drake
,
S. H.
, and
Meyer
,
A. A.
,
1998
, “
Augmented Reality Visualization for Laparoscopic Surgery
,”
International Conference on Medical Image Computing and Computer-Assisted Intervention
,
Cambridge, MA
,
Oct. 11–13
, Springer, pp.
934
943
.
19.
Schall
,
G.
,
Mendez
,
E.
,
Kruijff
,
E.
,
Veas
,
E.
,
Junghanns
,
S.
,
Reitinger
,
B.
, and
Schmalstieg
,
D.
,
2009
, “
Handheld Augmented Reality for Underground Infrastructure Visualization
,”
Pers. Ubiquitous Comput.
,
13
, pp.
281
291
.
20.
Marvel
,
J. A.
, and
van Wyk
,
K.
,
2016
, “
Simplified Framework for Robot Coordinate Registration for Manufacturing Applications
,”
2016 IEEE International Symposium on Assembly and Manufacturing (ISAM)
,
Fort Worth, TX
,
Aug. 21–22
, IEEE, pp.
56
63
.
21.
Zeng
,
T.
,
Li
,
C.
,
Liu
,
Q.
, and
Chen
,
X.
,
2014
, “
Tracking With Nonlinear Measurement Model by Coordinate Rotation Transformation
,”
Sci. China: Technol. Sci.
,
57
, pp.
2396
2406
.
22.
Universal Robots
,
2022
, “
Technical Specifications UR10
,” https://www.universal-robots.com/media/50880/ur10_bz.pdf, Accessed July 10, 2022.
23.
OptiTrack
,
2022
, “V120:Trio,” https://optitrack.com/cameras/v120-trio/specs.html, Accessed July 10, 2022.
24.
Saunier
,
N.
, and
Sayed
,
T.
,
2006
, “
A Feature-Based Tracking Algorithm for Vehicles in Intersections
,”
The 3rd Canadian Conference on Computer and Robot Vision (CRV’06)
,
Quebec, Canada
,
June 7–9
, IEEE, p.
59
.
25.
Nguyen
,
V.
, and
Marvel
,
J.
,
2022
, “
Evaluation of Data-Driven Models in Human-Robot Load-Sharing
,”
International Manufacturing Science and Engineering Conference
,
West Lafayette, IN
,
June 27–July 1
.
26.
Princely
,
F.
, and
Selvaraj
,
T.
,
2014
, “
Vision Assisted Robotic Deburring of Edge Burrs in Cast Parts
,”
Procedia Eng.
,
97
, pp.
1906
1914
.
You do not currently have access to this content.