This paper introduces a 3D object tracking method for an augmented reality (AR) assembly assistance application. The tracking method relies on point clouds; it uses 3D feature descriptors and point cloud matching with the iterative closest points (ICP) algorithm. The feature descriptors identify an object in a point cloud; ICP align a reference object with this point cloud. The challenge is to achieve high fidelity while maintaining camera frame rates. The point cloud and reference object sampling density are one of the key factors to meet this challenge. In this research, three-point sampling methods and two-point cloud search algorithms were compared to assess their fidelity when tracking typical products of mechanical engineering. The results indicate that a uniform sampling maintains the best fidelity at camera frame rates.

References

References
1.
Azuma
,
R.
,
1997
, “
A Survey of Augmented Reality
,”
Presence Teleoperators Virtual Environ.
,
6
(
2–3
), pp.
355
385
.
2.
Radkowski
,
R.
,
Herrema
,
J.
, and
Oliver
,
J.
,
2015
, “
Augmented Reality Based Manual Assembly Support With Visual Features for Different Degree of Difficulty
,”
J. Hum. Comput. Interact.
,
31
(
5
), pp.
337
349
.
3.
Henderson
,
S.
, and
Feiner
,
S.
,
2011
, “
Augmented Reality in the Psychomotor Phase of a Procedural Task
,”
2011 10th IEEE International Symposium on Mixed and Augmented Reality
, pp.
191
200
.
4.
Hou
,
L.
, and
Wang
,
X.
,
2013
, “
A Study on the Benefits of Augmented Reality in Retaining Working Memory in Assembly Tasks: A Focus on Differences in Gender
,”
Autom. Constr.
,
32
(
2013
), pp.
38
45
.
5.
Westerfield
,
G.
,
Mitrovic
,
A.
, and
Billinghurst
,
M.
,
2013
, “
Intelligent Augmented Reality Training for Assembly Tasks
,”
16th International Conference on Artificial Intelligence in Education (AIED 2013)
, H. Chad, Land, K. Yacef, J. Mostow, and P. Pavlik, eds., pp.
542
551
.
6.
Peniche
,
A.
,
Treffetz
,
H.
,
Diaz
,
C.
, and
Paramo
,
G.
,
2012
, “
Combing Virtual and Augmented Reality to Improve the Mechanical Assembly Training Process in Manufacturing
,”
American Conference on Applied Mathematics
, pp.
292
297
.
7.
Yuan
,
M. L.
,
Ong
,
S. K.
, and
Nee
,
A. Y. C.
,
2008
, “
Augmented Reality for Assembly Guidance Using a Virtual Interactive Tool
,”
Int. J. Prod. Res.
,
46
(
7
), pp.
1745
1767
.
8.
Wiedenmaier
,
S.
,
Oehme
,
O.
,
Schmidt
,
L.
, and
Luczak
,
H.
,
2003
, “
Augmented Reality (AR) for Assembly Processes Design and Experimental Evaluation
,”
Int. J. Hum. Comput. Interact.
,
16
(
3
), pp.
497
514
.
9.
Kato
,
H.
, and
Billinghurst
,
M.
,
1999
, “
Marker Tracking and HMD Calibration for a Video-Based Augmented Reality Conferencing System
,”
2nd International Workshop on Augmented Reality (IWAR 99)
, pp.
85
94
.
10.
Neumann
,
U.
, and
You
,
S.
,
1998
, “
Natural Feature Tracking for Augmented-Reality
,”
IEEE Transactions on Multimedia
, pp.
53
64
.
11.
Salas-Moreno
,
R. F.
,
Newcombe
,
R. A.
,
Strasdat
,
H.
,
Kelly
,
P. H. J.
, and
Davison
,
A. J.
,
2013
, “
Slam++: Simultaneous Localisation and Mapping at the Level of Objects
,”
2013 IEEE Conference Computer Vision and Pattern Recognition (CVPR)
, pp.
1352
1359
.
12.
Radkowski
,
R.
, and
Oliver
,
J.
,
2013
, “
Natural Feature Tracking Augmented Reality for On-Site Assembly Assistance Systems
,”
Human Computer Interaction International Conference
, pp.
281
290
.
13.
Besl
,
P.
, and
McKay
,
N.
,
1992
, “
A Method for Registration of 3D-Shapes
,”
PAMI.
,
14
(
2
), pp.
239
256
.
14.
Raczynski
,
A.
, and
Gussmann
,
P.
,
2004
, “
Services and Training Through Augmented Reality
,”
1st European Conference on Visual Media Production (CVMP’04)
, pp.
263
271
.
15.
Zenati
,
N.
,
Zerhouni
,
N.
, and
Achour
,
K.
,
2004
, “
Assistance to Maintenance in Industry Process Using an Augmented Reality System
,”
2004 IEEE international Conference on Industrial Technology (KIT)
, Vol. 2, pp.
848
852
.
16.
Reiners
,
D.
,
Stricker
,
D.
,
Klinker
,
G.
, and
Müller
,
S.
,
1998
, “
Augmented Reality for Construction Tasks: Doorlock Assembly
,”
1st International Workshop on Augmented Reality (IWAR 98)
, pp.
31
46
.
17.
Neumann
,
U.
, and
Majoros
,
A.
,
1998
, “
Cognitive, Performance, and Systems Issues for Augmented Reality Applications in Manufacturing and Maintenance
,”
Virtual Reality Annual International Symposium
, pp.
4
11
.
18.
Robertson
,
C.
,
MacIntyre
,
B.
, and
Walker
,
B.
,
2008
, “
An Evaluation of Graphical Context When the Graphics Are Outside of the Task Area
,”
7th IEEE and ACM International Symposium on Mixed and Augmented Reality (ISMAR 2008)
,
IEEE Computer Society
,
Washington, DC
, pp.
73
76
.
19.
Khuong
,
B. M.
,
Kiyokawa
,
K.
,
Mashita
,
T.
,
Miller
,
A.
,
Takemura
,
H.
, and
La Viola
,
J. J.
,
2014
, “
The Affectiveness of an AR-Based Context-Aware Assembly Support System in Object Assembly
,”
IEEE Virtual Reality (VR)
, pp.
57
62
.
20.
Radkowski
,
R.
, and
Oliver
,
J.
,
2014
, “
Monocular Depth Cues for Augmented Reality Applications to Enhance Spatial Perception Tasks
,”
Tools and Methods of Competitive Engineering (TMCE 2014)
, pp.
749
768
.
21.
Wang
,
Y.
,
Shen
,
Y.
,
Liu
,
D.
,
Wei
,
S.
, and
Zhu
,
C.
,
2010
, “
Key Technique of Assembly System in an Augmented Reality Environment
,”
2010 Second International Conference on Computer Modeling and Simulation
, pp.
133
137
.
22.
Song
,
J.
,
Jian
,
Q.
,
Sun
,
H.
, and
Gao
,
X.
,
2009
, “
Study of the Perception Mechanisms and Method of Virtual and Real Objects in Augmented Reality Assembly Environments
,”
4th IEEE Conference on Industrial Electronics and Applications
, pp.
1452
1456
.
23.
Baird
,
K. M.
, and
Barfield
,
W.
,
1999
, “
Evaluating the Effectiveness of Augmented Reality Displays for a Manual Assembly Task
,”
Virtual Reality
,
4
(
4
), pp.
250
259
.
24.
Korn
,
O.
,
Schmidt
,
A.
, and
Hörz
,
T.
,
2013
, “
Augmented Manufacturing: A Study With Impaired Persons on Assistive Systems Using In-Situ Projection
,”
6th International Conference on Pervasive Technologies Related to Assistive Environment (PETRA 2013)
, pp.
21:1
21:8
.
25.
Tang
,
A.
,
Owen
,
C.
,
Biocca
,
F.
, and
Mou
,
W.
,
2003
, “
Comparative Effectiveness of Augmented Reality in Object Assembly
,”
Conference on Human Factors in Computing Systems (CHI’03)
, pp.
73
80
.
26.
Webel
,
S.
,
Bockholt
,
U.
,
Engelke
,
T.
,
Gavish
,
N.
,
Olbrich
,
M.
, and
Preusche
,
C.
,
2011
, “
An Augmented Reality Training Platform for Assembly and Maintenance Skills
,”
J. Rob. Auton. Syst.
,
61
(
4
), pp.
398
403
.
27.
Nilsson
,
S.
, and
Johansson
,
B.
,
2007
, “
Fun and Usable: Augmented Reality Instructions in a Hospital Setting
,”
19th Australasian Conference on Computer-Human Interaction (OZCHI 07)
, pp.
123
130
.
28.
Miller
,
A.
,
White
,
B.
,
Charbonneau
,
E.
,
Kanzler
,
Z.
, and
LaViola
,
J. J.
, Jr.
,
2012
, “
Interactive 3D Model Acquisition and Tracking of Building Block Structures
,”
IEEE Trans. Visualization Comput. Graphics
,
18
(
4
), pp.
651
659
.
29.
Gupta
,
A.
,
Fox
,
D.
,
Curless
,
B.
, and
Cohen
,
M.
,
2012
, “
Duplotrack: A Real-Time System for Authoring and Guiding Duplo Block Assembly
,”
25th Annual ACM Symposium on User Interface Software and Technology
, pp.
389
402
.
30.
Newcombe
,
R. A.
,
Izadi
,
S.
,
Hilliges
,
O.
,
Molyneaux
,
D.
,
Kim
,
D.
,
Davison
,
A. J.
,
Kohli
,
P.
,
Shotton
,
J.
,
Hodges
,
S.
, and
Fitzgibbon
,
A.
,
2011
, “
Kinectfusion: Real-Time Dense Surface Mapping and Tracking
,”
International Symposium on Mixed and Augmented Reality
, pp.
127
136
.
31.
Garrett
,
T.
,
Debernardis
,
S.
,
Radkowski
,
R.
,
Chang
,
C. K.
,
Fiorentino
,
M.
,
Uva
,
A. E.
, and
Oliver
,
J.
,
2014
, “
Rigid-Object Tracking for Real-Time Augmented Reality
,”
ASME 2014 International Design Engineering Technical Conferences & Computers and Information in Engineering Conference
, pp.
1
10
.
32.
Rusinkiewicz
,
S.
, and
Levoy
,
M.
,
2001
, “
Efficient Variants of the ICP Algorithm
,”
IEEE International Workshop on 3D Digital Imaging and Modeling (3DIM)
, pp.
145
152
.
33.
Drost
,
B.
,
Ulrich
,
M.
,
Navab
,
N.
, and
Ilic.
,
S.
,
2010
, “
Model Globally, Match Locally: Efficient and Robust 3D Object Recognition
,”
IEEE Conference on Computer Vision and Pattern Recognition (CVPR)
, pp.
998
1005
.
34.
Turk
,
G.
, and
Levoy
,
M.
,
1994
, “
Zippered Polytgon Meshes From Range Images
,”
SIGGRAPH '94
, Computer Graphics Proceedings, pp.
311
318
.
35.
Masuda
,
T.
,
Sakaue
,
K.
, and
Yokoya
,
N.
,
1996
, “
Registration and Integration of Multiple Range Images for 3D-Model Construction
,”
CVPR
, pp.
879
883
.
36.
Greenspan
,
M.
, and
Yurick
,
M.
,
2003
, “
Approximate k-d Tree Search for Efficient ICP
,”
Fourth International Conference on 3-D Digital Imaging and Modeling (3DIM 2003)
, pp.
442
448
.
37.
Godin
,
G.
,
Rioux
,
M.
, and
Baribeau
,
R.
,
1994
, “
Three-Dimensional Registration Using Range and Intensity Information
,”
Proc. SPIE
2350, Videometrics III, pp.
279
290
.
38.
Zhang
,
Z.
,
1994
, “
Iterative Point Matching for Registration of Free-Form Curves and Surfaces
,”
Int. J. Comput. Vision
,
13
(
2
), pp.
119
152
.
39.
Arun
,
K.
,
Huang
,
T. S.
, and
Blostein
,
S. D.
,
1987
, “
Least-Squares Fitting of Two 3-D Point Sets
,”
IEEE Trans. Pattern Anal. Mach. Intell.
,
PAMI-9
(
5
), pp.
698
700
.
You do not currently have access to this content.