Abstract

The human performance benefits of four types of guided assembly instructions, including the Microsoft HoloLens 1, were analyzed in the context of a realistic assembly task. Several studies have confirmed the benefits of using augmented reality (AR) work instructions over traditional digital or paper instructions, but few have compared the effects of different AR hardware, including head-mounted displays, for complex assembly tasks. Participants completed a mock wing assembly task using the Microsoft HoloLens 1, and completion time, error count, and net promoter score (NPS) were recorded. These data were compared to data from previous research which employed desktop model-based instructions (MBIs), tablet MBI, and tablet AR instructions for the same task. The use of HoloLens AR instructions led to time saving of 16% over the tablet AR instructions. HoloLens users also had lower error rates than non-AR users. Despite the performance benefits of the HoloLens AR instructions, this condition had a lower NPS than the tablet AR group. The qualitative data showed that some users thought the HoloLens device was uncomfortable and that the tracking was not always exact. Although the users favored the tablet AR condition, the HoloLens condition had significantly faster assembly times. The authors recommend using the HoloLens for complex guided assembly instructions with minor changes, such as allowing the user to toggle the AR instructions on and off at will. The results of this paper can help manufacturing stakeholders understand the benefits of different AR technologies for manual assembly tasks.

References

References
1.
Liao
,
T.
, and
Humphreys
,
L.
,
2015
, “
Layar-Ed Places: Using Mobile Augmented Reality to Tactically Reengage, Reproduce, and Reappropriate Public Space
,”
New Media Soc.
,
17
(
9
), pp.
1418
1435
. 10.1177/1461444814527734
2.
Boeing
,
2018
, “
Boeing Tests Augmented Reality in the Factory
,” Boeing [Online], http://www.boeing.com/features/2018/01/augmented-reality-01-18.page, Accessed February 28, 2018.
3.
Palmarini
,
R.
,
Erkoyuncu
,
J. A.
,
Roy
,
R.
, and
Torabmostaedi
,
H.
,
2018
, “
A Systematic Review of Augmented Reality Applications in Maintenance
,”
Rob. Comput. Integr. Manuf.
,
49
, pp.
215
228
. 10.1016/j.rcim.2017.06.002
4.
Caudell
,
T. P.
, and
Mizell
,
D. W.
,
1992
, “
Augmented Reality: An Application of Heads-Up Display Technology to
,”
Proceedings of the Twenty-Fifth Hawaii International Conference on System Sciences
,
Kauai, HI
,
Jan. 7–10
, Vol.
2
,
IEEE
, pp.
659
669
.
5.
Wiedenmaier
,
S.
,
Oehme
,
O.
,
Schmidt
,
L.
, and
Luczak
,
H.
,
2003
, “
Augmented Reality (AR) for Assembly Processes Design and Experimental Evaluation
,”
Int. J. Hum. Comput. Interact.
,
16
(
3
), pp.
497
514
. 10.1207/S15327590IJHC1603_7
6.
Echtler
,
F.
,
Sturm
,
F.
,
Kindermann
,
K.
,
Klinker
,
G.
,
Stilla
,
J.
,
Trilk
,
J.
, and
Najafi
,
H.
,
2004
, “The Intelligent Welding Gun: Augmented Reality for Experimental Vehicle Construction,”
Virtual and Augmented Reality Applications in Manufacturing
,
S. K.
Ong
, and
A. Y. C.
Nee
, eds.,
Springer London
,
London, UK
, pp.
333
360
.
7.
Nilsson
,
S.
, and
Johansson
,
B.
,
2007
, “
Fun and Usable
,”
Proceedings of the 2007 Conference of the Computer–Human Interaction Special Interest Group (CHISIG) of Australia on Computer–Human Interaction: Design: Activities, Artifacts and Environments—OZCHI ‘07
,
Adelaide, Australia
,
Nov. 28–30
,
ACM Press
,
New York
, p.
123
.
8.
Henderson
,
S.
, and
Feiner
,
S.
,
2011
, “
Augmented Reality in the Psychomotor Phase of a Procedural Task
,”
2011 10th IEEE International Symposium on Mixed and Augmented Reality
,
Basel, Switzerland
,
Oct. 26–29
,
IEEE
, pp.
191
200
.
9.
Tang
,
A.
,
Owen
,
C.
,
Biocca
,
F.
, and
Mou
,
W.
,
2003
, “
Comparative Effectiveness of Augmented Reality in Object Assembly
,”
Proceedings of the SIGCHI Conference on Human Factors in Computing Systems
,
Fort Lauderdale, FL
,
Apr. 5–10
, pp.
73
80
.
10.
Richardson
,
T.
,
Gilbert
,
S.
,
Holub
,
J.
,
Macallister
,
A.
,
Radkowski
,
R.
,
Davies
,
P.
, and
Terry
,
S.
,
2014
, “
Fusing Self-Reported and Sensor Data From Mixed-Reality Training
,”
Interservice/Industry Training, Simulation and Education Conference (I/ITSEC)
,
Orlando, FL
,
Dec. 1–4
, pp.
1
12
,
Paper No. 14158
.
11.
Tatić
,
D.
, and
Tešić
,
B.
,
2017
, “
The Application of Augmented Reality Technologies for the Improvement of Occupational Safety in an Industrial Environment
,”
Comput. Ind.
,
85
, pp.
1
10
. 10.1016/j.compind.2016.11.004
12.
Azuma
,
R.
,
1997
, “
A Survey of Augmented Reality
,”
Presence
,
6
(
4
), pp.
355
385
.
13.
Azuma
,
R.
,
Baillot
,
Y.
, and
Behringer
,
R.
,
2001
, “
Recent Advances in Augmented Reality
,”
IEEE Comput. Graph. Appl.
,
21
(
6
), pp.
34
47
.
14.
Boud
,
A.
,
Haniff
,
D.
,
Baber
,
C.
, and
Steiner
,
S.
,
1999
, “
Virtual Reality and Augmented Reality as a Training Tool for Assembly Tasks
,”
IEEE International Conference Information Visualization
,
London, UK
,
July 14–16
, pp.
32
36
.
15.
Henderson
,
S.
, and
Feiner
,
S.
,
2009
, “
Evaluating the Benefits of Augmented Reality for Task Localization in Maintenance of an Armored Personnel Carrier Turret
,”
8th IEEE International Symposium on Mixed and Augmented Reality, 2009
,
Orlando, FL
,
Oct. 19–22
.
16.
Hou
,
L.
,
Wang
,
X.
, and
Truijens
,
M.
,
2015
, “
Using Augmented Reality to Facilitate Piping Assembly: An Experiment-Based Evaluation
,”
J. Comput. Civil Eng.
,
29
(
1
), p.
05014007
. 10.1061/(ASCE)CP.1943-5487.0000344
17.
Sanna
,
A.
,
Manuri
,
F.
,
Lamberti
,
F.
,
Member
,
S.
,
Paravati
,
G.
, and
Pezzolla
,
P.
,
2015
, “
Using Handheld Devices to Support Augmented Reality-Based Maintenance and Assembly Tasks
,”
IEEE International Conference on Consumer Electronics Using
,
Las Vegas, NV
,
Jan. 9–12
, pp.
178
179
.
18.
Nee
,
A. Y. C.
,
Ong
,
S. K.
,
Chryssolouris
,
G.
, and
Mourtzis
,
D.
,
2012
, “
Augmented Reality Applications in Design and Manufacturing
,”
CIRP Ann.
,
61
(
2
), pp.
657
679
. 10.1016/j.cirp.2012.05.010
19.
Biocca
,
F. A.
, and
Rolland
,
J. P.
,
1998
, “
Virtual Eyes Can Rearrange Your Body: Adaptation to Visual Displacement in See-Through, Head-Mounted Displays
,”
Presence
,
7
(
3
), pp.
262
277
. 10.1162/105474698565703
20.
Cuervo
,
E.
,
2017
, “
Beyond Reality
,”
GetMobile Mob. Comput. Commun.
,
21
(
2
), pp.
9
15
. 10.1145/3131214.3131218
21.
Feiner
,
S.
,
Macintyre
,
B.
, and
Seligmann
,
D.
,
1993
, “
Knowledge-Based Augmented Reality
,”
Commun. ACM
,
36
(
7
), pp.
53
62
. 10.1145/159544.159587
22.
De Crescenzio
,
F.
,
Fantini
,
M.
,
Persiani
,
F.
,
Di Stefano
,
L.
,
Azzari
,
P.
, and
Salti
,
S.
,
2011
, “
Augmented Reality for Aircraft Maintenance Training and Operations Support
,”
IEEE Comput. Graph. Appl.
,
31
(
1
), pp.
96
101
. 10.1109/MCG.2011.4
23.
Microsoft
,
2018
, “Why HoloLens,” [Online], https://www.microsoft.com/en-us/hololens/why-hololens, Accessed February 26, 2018.
24.
Loch
,
F.
,
Quint
,
F.
, and
Brishtel
,
I.
,
2016
, “
Comparing Video and Augmented Reality Assistance in Manual Assembly
,”
Proceedings—12th International Conference on Intelligent Environments, IE 2016
,
London, UK
,
Sept. 14–16
, pp.
147
150
.
25.
Doshi
,
A.
,
Smith
,
R. T.
,
Thomas
,
B. H.
, and
Bouras
,
C.
,
2017
, “
Use of Projector Based Augmented Reality to Improve Manual Spot-Welding Precision and Accuracy for Automotive Manufacturing
,”
Int. J. Adv. Manuf. Technol.
,
89
(
5–8
), p.
1288
. 10.1007/s00170-016-9164-5
26.
Uva
,
A. E.
,
Gattullo
,
M.
,
Manghisi
,
V. M.
,
Spagnulo
,
D.
,
Cascella
,
G. L.
, and
Fiorentino
,
M.
,
2017
, “
Evaluating the Effectiveness of Spatial Augmented Reality in Smart Manufacturing: A Solution for Manual Working Stations
,”
Int. J. Adv. Manuf. Technol.
,
94
(
1–4
), pp.
509
521
. 10.1007/s00170-017-0846-4
27.
Zheng
,
X. S.
,
Foucault
,
C.
,
Matos
,
P.
,
Silva
,
D.
,
Dasari
,
S.
,
Yang
,
T.
, and
Goose
,
S.
,
2015
, “
Eye-Wearable Technology for Machine Maintenance: Effects of Display Position and Hands-Free Operation
,”
Proceedings of the 33rd Annual ACM Conference Human Factors in Computing Systems
,
Seoul, South Korea
,
Apr. 18–23
, pp.
2125
2134
.
28.
Baird
,
K. M.
, and
Barfield
,
W.
,
1999
, “
Evaluating the Effectiveness of Augmented Reality Displays for a Manual Assembly Task
,”
Virtual Real.
,
4
(
4
), pp.
250
259
. 10.1007/BF01421808
29.
Wickens
,
C.
,
Gordon
,
S.
,
Liu
,
Y.
, and
Lee
,
J.
,
1998
,
An Introduction to Human Factors Engineering
,
Pearson Education, Inc.
,
New Jersey
.
30.
Neumann
,
U.
, and
Majoros
,
A.
,
1998
, “
Cognitive, Performance, and Systems Issues for Augmented Reality Applications in Manufacturing and Maintenance
,”
Proceedings. IEEE 1998 Virtual Reality Annual International Symposium (Cat. No. 98CB36180)
,
Atlanta, GA
,
Mar. 14–18
, pp.
4
11
.
31.
Funk
,
M.
,
Kosch
,
T.
, and
Schmidt
,
A.
,
2016
, “
Interactive Worker Assistance: Comparing the Effects of In-Situ Projection, Head-Mounted Displays, Tablet, and Paper Instructions
,”
Proceedings of the 2016 ACM International Joint Conference on Pervasive and Ubiquitous Computing
,
Heidelburg, Germany
,
Sept. 12–16
, ACM, New York, pp.
934
939
.
32.
Funk
,
M.
,
Kosch
,
T.
,
Greenwald
,
S. W.
, and
Schmidt
,
A.
,
2015
, “
A Benchmark for Interactive Augmented Reality Instructions for Assembly Tasks
,”
Proceedings of the 14th International Conference on Mobile and Ubiquitous Multimedia—MUM ‘15
,
Linz, Austria
,
Nov. 30–Dec. 2
, ACM Press, New York, pp.
253
257
.
33.
Funk
,
M.
,
Mayer
,
S.
,
Nistor
,
M.
, and
Schmidt
,
A.
,
2016
, “
Mobile In-Situ Pick-by-Vision
,”
Proceedings of the 9th ACM International Conference on PErvasive Technologies Related to Assistive Environments—PETRA ‘16
,
Corfu, Greece
,
June 29–July 1
, ACM Press, New York, pp.
1
4
.
34.
Blattgerste
,
J.
,
Strenge
,
B.
,
Renner
,
P.
,
Pfeiffer
,
T.
, and
Essig
,
K.
,
2017
, “
Comparing Conventional and Augmented Reality Instructions for Manual Assembly Tasks
,”
Proceedings of the 10th International Conference on PErvasive Technologies Related to Assistive Environments—PETRA ‘17
,
Island of Rhodes, Greece
,
June 21–23
, pp.
75
82
.
35.
Syberfeldt
,
A.
,
Danielsson
,
O.
,
Holm
,
M.
, and
Wang
,
L.
,
2015
, “
Visual Assembling Guidance Using Augmented Reality
,”
Procedia Manuf.
,
1
, pp.
98
109
. 10.1016/j.promfg.2015.09.068
36.
Kantowitz
,
B. H.
,
1987
, “Mental Workload,”
Human Factors Psychology
,
P. A.
Hancock
, ed.,
North-Holland
,
Amsterdam, Netherlands
, pp.
81
121
.
37.
MacAllister
,
A.
,
Hoover
,
M.
,
Gilbert
,
S.
,
Oliver
,
J.
,
Radkowski
,
R.
,
Garrett
,
T.
,
Holub
,
J.
,
Winer
,
E.
,
Terry
,
S.
, and
Davies
,
P.
,
2017
, “
Comparing Visual Assembly Aids for Augmented Reality Work Instructions
,”
Interservice/Industry Training, Simulation, and Education Conference (I/ITSEC)
,
Orlando, FL
,
Nov. 27–Dec. 1
.
38.
Microsoft
, “
Install the Tools—Mixed Reality|Microsoft Docs
” [Online], https://docs.microsoft.com/en-us/windows/mixed-reality/install-the-tools#mixed-reality-toolkit, Accessed April 4, 2018.
39.
Träskbäack
,
M.
, and
Haller
,
M.
,
2004
, “
Mixed Reality Training Application for an Oil Refinery
,”
Proceedings of the 2004 ACM SIGGRAPH International Conference on Virtual Reality Continuum and Its Applications in Industry—VRCAI ‘04
,
Singapore
,
June 16–18
,
ACM Press
,
New York
, p.
324
.
40.
Radkowski
,
R.
,
Herrema
,
J.
, and
Oliver
,
J.
,
2015
, “
Augmented Reality-Based Manual Assembly Support With Visual Features for Different Degrees of Difficulty
,”
Int. J. Hum. Comput. Interact.
,
31
(
5
), pp.
337
349
. 10.1080/10447318.2014.994194
41.
Schwerdtfeger
,
B.
, and
Klinker
,
G.
,
2008
, “
Supporting Order Picking With Augmented Reality
,”
2008 7th IEEE/ACM International Symposium on Mixed and Augmented Reality
,
Cambridge, UK
,
Sept. 15–18
, IEEE, pp.
91
94
.
42.
Miller
,
J.
,
Hoover
,
M.
, and
Winer
,
E.
,
2019
, “
Overcoming Limitations of the HoloLens For Use in Product Assembly
,”
IS&T International Symposium on Electronic Imaging Science and Technology.
,
San Francisco, CA
,
Jan. 13–17
.
43.
Ekstrom
,
R.
,
French
,
J.
,
Harman
,
H.
, and
Dermen
,
D.
,
1976
,
Manual for Kit of Factor Referenced Cognitive Tests
,
Educational Testing Service
,
Princeton, NJ
.
44.
Macallister
,
A.
,
Gilbert
,
S.
,
Holub
,
J.
,
Winer
,
E.
, and
Davies
,
P.
,
2016
, “
Comparison of Navigation Methods in Augmented Reality Guided Assembly
,”
Interservice/Industry Training, Simulation, Education Conference (I/ITSEC)
,
Orlando, FL
,
Nov. 28–Dec. 2
, pp.
1
14
.
45.
Hoover
,
M.
,
MacAllister
,
A.
,
Holub
,
J.
,
Gilbert
,
S.
,
Winer
,
E.
, and
Davies
,
P.
,
2016
, “
Assembly Training Using Commodity Physiological Sensors
,”
Interservice/Industry Training, Simulation, and Education Conference (I/ITSEC).
,
Orlando, FL
,
Nov. 28–Dec. 2
.
46.
Reichheld
,
F.
,
2003
, “
The One Number You Need to Grow
,”
Harv. Bus. Rev.
,
81
(
12
), pp.
46
54
.
47.
Creswell
,
J.
, and
Creswell
,
D.
,
2017
,
Research Design: Qualitative, Quantitative, and Mixed Methods Approaches
,
Sage
,
Los Angeles, CA
.
You do not currently have access to this content.