Modern color and depth (RGB-D) sensing systems are capable of reconstructing convincing virtual representations of real world environments. These virtual reconstructions can be used as the foundation for virtual reality (VR) and augmented reality environments due to their high-quality visualizations. However, a main limitation of modern virtual reconstruction methods is the time it takes to incorporate new data and update the virtual reconstruction. This delay prevents the reconstruction from accurately rendering dynamic objects or portions of the environment (like an engineer performing an inspection of a machinery or laboratory space). The authors propose a multisensor method to dynamically capture objects in an indoor environment. The method automatically aligns the sensors using modern image homography techniques, leverages graphics processing units (GPUs) to process the large number of independent RGB-D data points, and renders them in real time. Incorporating and aligning multiple sensors allows a larger area to be captured from multiple angles, providing a more complete virtual representation of the physical space. Performing processing on GPU's leverages the large number of processing cores available to minimize the delay between data capture and rendering. A case study using commodity RGB-D sensors, computing hardware, and standard transmission control protocol internet connections is presented to demonstrate the viability of the proposed method.

References

References
1.
Cutting
,
J. E.
,
1997
, “
How the Eye Measures Reality and Virtual Reality
,”
Behav. Res. Methods, Instrum., Comput.
,
29
(
1
), pp.
27
36
.
2.
Bowman
,
D. A.
, and
McMahan
,
R. P.
,
2007
, “
Virtual Reality: How Much Immersion is Enough?
,”
Computer
,
40
(
7
), pp.
36
43
.
3.
Lee
,
K. M.
,
2004
, “
Why Presence Occurs: Evolutionary Psychology, Media Equation, and Presence
,”
Presence: Teleoperators Virtual Environ.
,
13
(
4
), pp.
494
505
.
4.
Turner
,
E.
,
Cheng
,
P.
, and
Zakhor
,
A.
,
2015
, “
Fast, Automated, Scalable Generation of Textured 3D Models of Indoor Environments
,”
IEEE J. Sel. Top. Signal Process.
,
9
(
3
), pp.
409
421
.
5.
Hamzeh
,
O.
, and
Elnagar
,
A.
,
2015
, “
A Kinect-Based Indoor Mobile Robot Localization
,”
Tenth International Symposium on Mechatronics and Its Applications
(
ISMA
), Sharjah, United Arab Emirates, Dec. 8–10, pp.
1
6
.
6.
Newcombe
,
R. A.
,
Izadi
,
S.
,
Hilliges
,
O.
,
Molyneaux
,
D.
,
Kim
,
D.
,
Davison
,
A.
,
Kohi
,
P.
,
Shotton
,
J.
,
Hodges
,
S.
, and
Fitzgibbon
,
A.
,
2011
, “
KinectFusion: Real-Time Dense Surface Mapping and Tracking
,”
Tenth IEEE International Symposium on Mixed and Augmented Reality
(
ISMAR
), Basel, Switzerland, Oct. 26–29, pp.
127
136
.
7.
Microsoft,
2011
, “
Kinect Fusion Explorer-WPF C# Sample
,” Microsoft Inc., Redmond, WA, accessed Feb. 16, 2017, https://msdn.microsoft.com/en-us/library/dn193975.aspx
8.
Lesniak
,
K.
,
Terpenny
,
J.
,
Tucker
,
C. S.
,
Anumba
,
C.
, and
Bilén
,
S. G.
,
2016
, “
Immersive Distributed Design Through Real-Time Capture, Translation, and Rendering of Three-Dimensional Mesh Data
,”
ASME J. Comput. Inf. Sci. Eng.
,
17
(
3
), p.
031010
.
9.
Oculus VR, LLC,
2016
, “
Oculus Rift|Oculus
,” Oculus VR, LLC, Menlo Park, CA, accessed Feb. 16, 2017, https://www.oculus.com/rift/#oui-csl-rift-games=mages-tale
10.
Ookla,
2016
, “
United States Speedtest Market Report
,” Ookla, Kalispell, MT, accessed Feb. 10, 2017, http://www.speedtest.net/reports/united-states/
11.
Yang
,
R. S.
,
Chan
,
Y. H.
,
Gong
,
R.
,
Nguyen
,
M.
,
Strozzi
,
A. G.
,
Delmas
,
P.
,
Gimel'farb
,
G.
, and
Ababou
,
R.
,
2013
, “
Multi-Kinect Scene Reconstruction: Calibration and Depth Inconsistencies
,”
28th International Conference on Image and Vision Computing New Zealand
(
IVCNZ
), Wellington, New Zealand, Nov. 27–29, pp.
47
52
.
12.
Asteriadis
,
S.
,
Chatzitofis
,
A.
,
Zarpalas
,
D.
,
Alexiadis
,
D. S.
, and
Daras
,
P.
,
2013
, “
Estimating Human Motion From Multiple Kinect Sensors
,”
Sixth International Conference on Computer Vision/Computer Graphics Collaboration Techniques and Applications
(MIRAGE), Berlin, June 6–7, Paper No.
3
.
13.
Harris
,
C.
, and
Stephens
,
M.
,
1988
, “
A Combined Corner and Edge Detector
,”
Alvey Vision Conference
, Manchester, UK, Aug. 31–Sept. 2, Paper No.
50
.http://citeseer.ist.psu.edu/viewdoc/download;jsessionid=8475C77EC4C2AD0EDC7C61C14D189E33?doi=10.1.1.231.1604&rep=rep1&type=pdf
14.
Dubrofsky
,
E.
,
2009
, “
Homography Estimation
,”
Master's thesis
,
Univerzita Britské Kolumbie
,
Vancouver, BC, Canada
.
15.
C. R. Souza,
2014
, “
Accord.NET Framework
,” São Carlos, Brazil, accessed Oct. 3, 2017, http://accord-framework.net/
16.
Ni
,
D.
,
Song
,
A.
,
Xu
,
X.
,
Li
,
H.
,
Zhu
,
C.
, and
Zeng
,
H.
,
2017
, “
3D-Point-Cloud Registration and Real-World Dynamic Modelling-Based Virtual Environment Building Method for Teleoperation
,”
Robotica
,
35
(
10
), pp.
1958
1974
.
17.
Kim
,
S.
, and
Park
,
J.
,
2017
, “
Robust Haptic Exploration of Remote Environments Represented by Streamed Point Cloud Data
,” IEEE
World Haptics Conference
(
WHC
), Munich, Germany, June 6–9, pp.
358
363
.
18.
Su
,
P.-C.
,
Xu
,
W.
,
Shen
,
J.
, and
Cheung
,
S. S.
,
2017
, “
Real-Time Rendering of Physical Scene on Virtual Curved Mirror With RGB-D Camera Networks
,”
IEEE International Conference on Multimedia & Expo Workshops
(
ICMEW
), Hong Kong, China, July 10–14, pp.
79
84
.
19.
Garrett
,
T.
,
Debernardis
,
S.
,
Oliver
,
J.
, and
Radkowski
,
R.
,
2016
, “
Poisson Mesh Reconstruction for Accurate Object Tracking With Low-Fidelity Point Clouds
,”
ASME J. Comput. Inf. Sci. Eng.
,
17
(
1
), p.
011003
.
20.
Epic Games,
2014
, “
What is Unreal Engine 4
,” Epic Games, Inc., Cary, NC, accessed Feb. 16, 2017, https://www.unrealengine.com/what-is-unreal-engine-4
21.
Sharples
,
S.
,
Cobb
,
S.
,
Moody
,
A.
, and
Wilson
,
J. R.
,
2008
, “
Virtual Reality Induced Symptoms and Effects (VRISE): Comparison of Head Mounted Display (HMD), Desktop and Projection Display Systems
,”
Displays
,
29
(
2
), pp.
58
69
.
22.
Microsoft,
2010
, “
GZipStream Class (System.IO.Compression)
,” Microsoft Inc., Redmond, WA, accessed Feb. 16, 2017, https://msdn.microsoft.com/en-us/library/system.io.compression.gzipstream(v=vs.110).aspx
You do not currently have access to this content.