Abstract

Extended reality (XR) technologies have realized significant value for design, manufacturing, and sustainment processes. However, industrial XR, or XR implemented within industrial applications, suffers from scalability and flexibility challenges due to fundamental gaps with interoperability between data, models, and workflows. Though there has been a number of recent efforts to improve the interoperability of industrial XR technologies, progress has been hindered by an innate separation between the domain-specific models (e.g., manufacturing execution data, material specifications, and product manufacturing information) with XR (often-standard) processes (e.g., multiscale spatial representations and data formats optimized for run-time presentation). In this paper, we elaborate on promising research directions and opportunities around which the manufacturing and visualization academic community can rally. To establish such research directions, we (1) conducted a meta-review on well-established state-of-the-art review articles that have already presented in-depth surveys on application areas for industrial XR, such as maintenance, assembly, and inspection and (2) mapped those findings to publicly published priorities from across the US Department of Defense. We hope that our presented research agenda will spur interdisciplinary work across academic silos, i.e., manufacturing and visualization communities, and engages either community within work groups led by the other, e.g., within standards development organizations.

References

1.
Lele
,
A.
,
2013
, “
Virtual Reality and Its Military Utility
,”
J. Ambient Intell. Humanized Comput.
,
4
(
1
), pp.
17
26
.
2.
Gilbert
,
A.
,
2016
, “
Augmented Reality for the US Air Force
,”
International Conference on Virtual, Augmented and Mixed Reality (VAMR 2016)
,
Toronto, Canada
,
July 17–22
, Springer, pp.
375
385
.
3.
Swann
,
D.
,
2005
, “Chapter 63,”
Geographical Information Systems: Principles, Techniques, Management and Applications, 2nd Edition, Abridged
, P. A. Longley, M. F. Goodchild, D. J. Maguire, and D. W. Rhind, eds., Vol.
2
,
Wiley
,
Hoboken, NJ
, pp.
889
899
.
4.
Milgram
,
P.
,
Takemura
,
H.
,
Utsumi
,
A.
, and
Kishino
,
F.
,
1995
, “
Augmented Reality: A Class of Displays on the Reality-Virtuality Continuum
,”
Proc. SPIE 2351, Telemanipulator and Telepresence Technologies
,
Boston, MA
,
Oct. 31–Nov. 4, 1994
, Vol. 2351, SPIE, pp.
282
292
.
5.
Lu
,
Y.
,
Morris
,
K. C.
, and
Frechette
,
S.
,
2016
, “
Current Standards Landscape for Smart Manufacturing Systems
,” National Institute of Standards and Technology, NISTIR, Gaithersburg, MD,
8107
(
3
).
6.
Perey
,
C.
,
2015
, “
Open and Interoperable Augmented Reality and the IEEE [standards]
,”
IEEE Consumer Electron. Mag.
,
4
(
4
), pp.
133
135
.
7.
MTConnect Institute, 2014, MTConnect Standard. Accessed on 31 March, 2017.
8.
ASME Y14.41-2012, 2012, Digital Product Definition Data Practices, American Society of Mechanical Engineers, New York.
9.
Bhatia
,
S.
,
Cozzi
,
P.
,
Knyazev
,
A.
, and
Parisi
,
T.
,
2021
, gltf 2.0 specification, Tech. Rep., Khronos Group, 2017, https://www.khronos.org/gltf/.
10.
Nee
,
A.
,
Ong
,
S.
,
Chryssolouris
,
G.
, and
Mourtzis
,
D.
,
2012
, “
Augmented Reality Applications in Design and Manufacturing
,”
CIRP Ann.
,
61
(
2
), pp.
657
679
.
11.
Fraga-Lamas
,
P.
,
FernáNdez-CaraméS
,
T. M.
,
Blanco-Novoa
,
O.
, and
Vilar-Montesinos
,
M. A.
,
2018
, “
A Review on Industrial Augmented Reality Systems for the Industry 4.0 Shipyard
,”
IEEE Access
,
6
(
1
), pp.
13358
13375
.
12.
Palmarini
,
R.
,
Erkoyuncu
,
J. A.
,
Roy
,
R.
, and
Torabmostaedi
,
H.
,
2018
, “
A Systematic Review of Augmented Reality Applications in Maintenance
,”
Rob. Comput.-Integr. Manuf.
,
49
(
1
), pp.
215
228
.
13.
Bottani
,
E.
, and
Vignali
,
G.
,
2019
, “
Augmented Reality Technology in the Manufacturing Industry: A Review of the Last Decade
,”
IISE Trans.
,
51
(
3
), pp.
284
310
.
14.
de Souza Cardoso
,
L. F.
,
Mariano
,
F. C. M. Q.
, and
Zorzal
,
E. R.
,
2020
, “
A Survey of Industrial Augmented Reality
,”
Comput. Ind. Eng.
,
139
(
1
), p.
106159
.
15.
Egger
,
J.
, and
Masood
,
T.
,
2020
, “
Augmented Reality in Support of Intelligent Manufacturing ‘A Systematic Literature Review’
,”
Comput. Ind. Eng.
,
140
(
1
), p.
106195
.
16.
Baroroh
,
D. K.
,
Chu
,
C. -H.
, and
Wang
,
L.
,
2021
, “
Systematic Literature Review on Augmented Reality in Smart Manufacturing: Collaboration Between Human and Computational Intelligence
,”
J. Manuf. Syst.
,
61
(
1
), pp.
696
711
.
17.
Janssen
,
M.
,
Estevez
,
E.
, and
Janowski
,
T.
,
2014
, “
Interoperability in Big, Open, and Linked Data—Organizational Maturity, Capabilities, and Data Portfolios
,”
Computer
,
47
(
10
), pp.
44
49
.
18.
Ray
,
S. R.
, and
Jones
,
A. T.
,
2006
, “
Manufacturing Interoperability
,”
J. Intell. Manuf.
,
17
(
6
), pp.
681
688
.
19.
Zeid
,
A.
,
Sundaram
,
S.
,
Moghaddam
,
M.
,
Kamarthi
,
S.
, and
Marion
,
T.
,
2019
, “
Interoperability in Smart Manufacturing: Research Challenges
,”
Machines
,
7
(
2
), p.
21
.
20.
Lu
,
Y.
,
Xu
,
X.
, and
Wang
,
L.
,
2020
, “
Smart Manufacturing Process and System Automation—A Critical Review of the Standards and Envisioned Scenarios
,”
J. Manuf. Syst.
,
56
(
1
), pp.
312
325
.
21.
Hedberg
,
T.
,
Lubell
,
J.
,
Fischer
,
L.
,
Maggiano
,
L.
, and
Barnard Feeney
,
A.
,
2016
, “
Testing the Digital Thread in Support of Model-Based Manufacturing and Inspection
,”
ASME J. Comput. Inf. Sci. Eng.
,
16
(
2
), p.
021001
.
22.
Scholz
,
J.
,
Bernstein
,
W. Z.
, and
Radkowski
,
R.
,
2022
, “
Research Directions for Merging Geospatial Technologies With Smart Manufacturing Systems
,”
Smart Sustain. Manuf. Syst.
,
6
(
1
), p.
226
.
23.
Autodesk, 2023, FBX – FilmBox 3D File Format.
24.
ARM Institute, 2022, “Project Highlight: Virtual Part Repair Programming for Robotic Thermal Spray Applications,” https://arminstitute.org/news/project-highlight-virutal-part-repair/.
25.
Kruijff
,
E.
,
Swan
,
J. E.
, and
Feiner
,
S.
,
2010
, “
Perceptual Issues in Augmented Reality Revisited
,”
2010 IEEE International Symposium on Mixed and Augmented Reality (ISMAR)
,
Seoul, South Korea
,
Oct. 13–16
, IEEE, pp.
3
12
.
26.
Elmqvist
,
N.
, and
Yi
,
J. S.
,
2012
, “
Patterns for Visualization Evaluation
,”
BELIV '12: Proceedings of the 2012 BELIV Workshop: Beyond Time and Errors – Novel Methods for Visualization
,
Seattle, WA
,
Oct. 14–15
, pp.
1
8
.
27.
Kemmerer
,
S. J.
,
1999
, “STEP: The Grand Experience,” NIST Pub Series – Special Publication (NIST SP).
28.
Epic Games, 2023, The Unreal Engine, https://www.unrealengine.com.
29.
Unity Technologies, 2023, Unity3D Game Engine, https://unity.com/.
30.
Perey
,
C.
, and
Bernstein
,
W. Z.
,
2022
, “
A Research Agenda for Enterprise Augmented Reality
,”
2022 IEEE Conference on Virtual Reality and 3D User Interfaces Abstracts and Workshops (VRW)
,
Christchurch, New Zealand
,
Mar. 12–16
, IEEE, pp.
477
480
.
31.
Vernica
,
T.
,
Lipman
,
R.
,
Kramer
,
T.
,
Kwon
,
S.
, and
Bernstein
,
W. Z.
,
2022
, “
Visualizing Standardized Model-Based Design and Inspection Data in Augmented Reality
,”
ASME J. Comput. Inf. Sci. Eng.
,
22
(
4
), p.
041001
.
32.
Mirzaiee
,
R.
,
Vernica
,
T.
,
Scheuringer
,
K.
, and
Bernstein
,
W. Z.
,
2022
, “
Towards Retargetable Animations for Industrial Augmented Reality
,”
2022 IEEE Conference on Virtual Reality and 3D User Interfaces Abstracts and Workshops (VRW)
,
Christchurch, New Zealand
,
Mar. 12–16
, IEEE, pp.
872
873
.
33.
Durscher
,
R.
,
Pankonien
,
A. M.
, and
Bhagat
,
N.
,
2019
, “
AArDVARK: Aerospace Analysis and Design in Virtual and Augmented Reality ToolKit
,”
Proceedings of the AIAA Aviation 2019 Forum
,
Dallas, TX
,
June 17–21
, p.
3560
.
34.
Metaverse Standards Forum, 2023, “Metaverse Standards Forum,” https://metaverse-standards.org/.
35.
Satyanarayanan
,
M.
,
2017
, “
The Emergence of Edge Computing
,”
Computer
,
50
(
1
), pp.
30
39
.
36.
Satyanarayanan
,
M.
,
Simoens
,
P.
,
Xiao
,
Y.
,
Pillai
,
P.
,
Chen
,
Z.
,
Ha
,
K.
,
Hu
,
W.
, and
Amos
,
B.
,
2015
, “
Edge Analytics in the Internet of Things
,”
IEEE Pervasive Comput.
,
14
(
2
), pp.
24
31
.
37.
Kwon
,
S.
,
Monnier
,
L. V.
,
Barbau
,
R.
, and
Bernstein
,
W. Z.
,
2020
, “
Enriching Standards-Based Digital Thread by Fusing As-Designed and As-Inspected Data Using Knowledge Graphs
,”
Adv. Eng. Inform.
,
46
(
1
), p.
101102
.
38.
Kulvatunyou
,
B.
,
Wallace
,
E.
,
Kiritsis
,
D.
,
Smith
,
B.
, and
Will
,
C.
,
2018
, “
The Industrial Ontologies Foundry Proof-of-Concept Project
,” Advances in Production Management Systems, Smart Manufacturing for Industry 4.0: IFIP WG 5.7 International Conference, APMS 2018, Seoul, Korea, Aug. 26–30, Proceedings, Part II,
Springer
, pp.
402
409
.
39.
Brundage
,
M. P.
,
Sexton
,
T.
,
Hodkiewicz
,
M.
,
Dima
,
A.
, and
Lukens
,
S.
,
2021
, “
Technical Language Processing: Unlocking Maintenance Knowledge
,”
Manuf. Lett.
,
27
(
1
), pp.
42
46
.
You do not currently have access to this content.