Recent development of per-frame motion extraction method can generate the skeleton of human motion in real-time with the help of RGB-D cameras such as Kinect. This leads to an economic device to provide human motion as input for real-time applications. As generated by a single-view image plus depth information, the extracted skeleton usually has problems of unwanted vibration, bone-length variation, self-occlusion, etc. This paper presents an approach to overcome these problems by synthesizing the skeletons generated by duplex Kinects, which capture the human motion in different views. The major technical difficulty of this synthesis comes from the inconsistency of two skeletons. Our algorithm is formulated under the constrained optimization framework by using the bone-lengths as hard constraints and the tradeoff between inconsistent joint positions as soft constraints. Schemes are developed to detect and re-position the problematic joints generated by per-frame method from duplex Kinects. As a result, we develop an easy, cheap and fast approach that can improve the skeleton of human motion at an average speed of 5 ms per frame.

References

References
1.
Cavazza
,
M.
,
Earnshaw
,
R.
,
Magnenat-Thalmann
,
N.
, and
Thalmann
,
D.
,
1998
, “
Motion Control of Virtual Humans
,”
IEEE Comput. Graph. Appl.
,
18
(
5
), pp.
24
31
.10.1109/38.708558
2.
Pan
,
Z.
,
Xu
,
W.
,
Huang
,
J.
,
Zhang
,
M.
, and
Shi
,
J.
,
2003
, “
Easybowling: A Small Bowling Machine Based on Virtual Simulation
,”
Comput. Graph.
,
27
, pp.
231
238
.10.1016/S0097-8493(02)00280-7
3.
Chan
,
J. C. P.
,
Leung
,
H.
,
Tang
,
J. K. T.
, and
Komura
,
T.
,
2011
, “
A Virtual Reality Dance Training System Using Motion Capture Technology
,”
IEEE Trans. Learn. Technol.
,
4
(
2
), pp.
187
195
.10.1109/TLT.2010.27
4.
Woltring
,
H. J.
,
1974
, “
New Possibilities for Human Motion Studies by Real-Time Light Spot Position Measurement
,”
Biotelemetry
,
1
(
3
), pp.
132
146
.
5.
Vicon
, “
Vicon Motion Capture System
,” available at: www.vicon.com
6.
Xsens
, “
Xsens MVN
,” available at: www.xsens.com
7.
Olson
,
E.
,
Leonard
,
J.
, and
Teller
,
S.
,
2006
, “
Robust Rangeonly Beacon Localization
,”
J. Ocean. Eng.
,
31
(
4
), pp.
949
958
.10.1109/JOE.2006.880386
8.
Hazas
,
M.
, and
Ward
,
A.
,
2002
, “
A Novel Broadband Ultrasonic Location System
,”
Proceedings of the 4th International Conference on Ubiquitous Computing
, pp.
264
280
.
9.
Motion
,
M.
, Gypsy, available at: http://www.motion-capture-system.com
10.
Measurand
Shapewrap, available at: http://www.metamotion.com
11.
Foxlin
,
E.
, and
Harrington
,
M.
,
2000
, “
Weartrack: A self-Referenced Head and Hand Tracker for Wearable Computers and Portable VR
,”
Proceedings of the 4th IEEE International Symposium on Wearable Computers
, pp.
155
162
.
12.
Bachmann
,
E. R.
,
McGhee
,
R. B.
,
Yun
,
X.
, and
Zyda
,
M. J.
,
2001
, “
Inertial and Magnetic Posture Tracking for Inserting Humans into Networked Virtual Environments
,”
Proceedings of the ACM Symposium on Virtual Reality Software and Technology
, pp.
9
16
.
13.
Vlasic
,
D.
,
Adelsberger
,
R.
,
Vannucci
,
G.
,
Barnwell
,
J.
,
Gross
,
M.
,
Matusik
,
W.
, and
Popović
,
J.
,
2007
, “
Practical Motion Capture in Everyday Surroundings
,”
ACM Trans. Graph.
,
26
(
3
), pp.
35:1
35:10
.10.1145/1276377.1276421
14.
Microsoft
, “
Microsoft Kinect for Windows SDK
,” www.microsoft.com
15.
Hauswiesner
,
S.
,
Straka
,
M.
, and
Reitmayr
,
G.
,
2011
, “
Free Viewpoint Virtual Try-On With Commodity Depth Cameras
,”
Proceedings of the 10th International Conference on Virtual Reality Continuum and Its Applications in Industry
, pp.
23
30
.
16.
Wilson
,
A. D.
, and
Benko
,
H.
,
2010
, “
Combining Multiple Depth Cameras and Projectors for Interactions on, Above and Between Surfaces
,”
Proceedings of the 23nd Annual ACM Symposium on User Interface Software and Technology
, pp.
273
282
.
17.
Newcombe
,
R. A.
,
Izadi
,
S.
,
Hilliges
,
O.
,
Molyneaux
,
D.
,
Kim
,
D.
,
Davison
,
A. J.
,
Kohli
,
P.
,
Shotton
,
J.
,
Hodges
,
S.
, and
Fitzgibbon
,
A.
,
2011
, “
Kinectfusion: Real-Time Dense Surface Mapping and Tracking
,”
Proceedings of the 2011 10th IEEE International Symposium on Mixed and Augmented Reality
, pp.
127
136
.
18.
Izadi
,
S.
,
Kim
,
D.
,
Hilliges
,
O.
,
Molyneaux
,
D.
,
Newcombe
,
R.
,
Kohli
,
P.
,
Shotton
,
J.
,
Hodges
,
S.
,
Freeman
,
D.
,
Davison
,
A.
, and
Fitzgibbon
,
A.
,
2011
, “
Kinectfusion: Real-Time 3d Reconstruction and Interaction Using a Moving Depth Camera
,”
Proceedings of the 24th Annual ACM Symposium on User Interface Software and Technology
, pp.
559
568
.
19.
Tong
,
J.
,
Zhou
,
J.
,
Liu
,
L.
,
Pan
,
Z.
, and
Yan
,
H.
,
2012
, “
Scanning 3D Full Human Bodies Using Kinects
,”
IEEE Trans. Vis. Comput. Graph.
,
18
, pp.
643
650
.10.1109/TVCG.2012.56
20.
Weiss
,
A.
,
Hirshberg
,
D.
, and
Black
,
M.
,
2011
, “
Home 3D Body Scans From Noisy Image and Range Data
,”
2011 IEEE International Conference on Computer Vision (ICCV)
, pp.
1951
1958
.
21.
Shotton
,
J.
,
Fitzgibbon
,
A.
,
Cook
,
M.
,
Sharp
,
T.
,
Finocchio
,
M.
,
Moore
,
R.
,
Kipman
,
A.
, and
Blake
,
A.
,
2011
, “
Real-Time Human Pose Recognition in Parts From Single Depth Images
,”
Proceedings of the 2011 IEEE Conference on Computer Vision and Pattern Recognition
, CVPR’11, IEEE Computer Society, pp.
1297
1304
.
22.
Activate3D
, “
Activate3D's Intelligent Character Motion (ICM)
,” available at: www.http://activate3d.com
23.
Moeslund
,
T. B.
,
Hilton
,
A.
, and
Krüger
, V
.
,
2006
, “
A Survey of Advances in Vision-Based Human Motion Capture and Analysis
,”
Comput. Vis. Image Underst.
,
104
(
2
), pp.
90
126
.10.1016/j.cviu.2006.08.002
24.
Poppe
,
R.
,
2007
, “
Vision-Based Human Motion Analysis: An Overview
,”
Comput. Vis. Image Underst.
,
108
(
1–2
), pp.
4
18
.10.1016/j.cviu.2006.10.016
25.
Rose
,
C.
,
Bodenheimer
,
B.
, and
Cohen
,
M. F.
,
1998
, “
Verbs and Adverbs: Multidimensional Motion Interpolation Using Radial Basis Functions
,”
IEEE Comput. Graph. Appl.
,
18
, pp.
32
40
.10.1109/38.708559
26.
Piazza
,
T.
,
Lundström
,
J.
,
Hugestrand
,
A.
,
Kunz
,
A.
, and
Fjeld
,
M.
,
2009
, “
Towards Solving the Missing Marker Problem in Realtime Motion Capture
,”
Proceedings of ASME 2009 IDETC/CIE Conference
, pp.
1521
1526
.
27.
Li
,
L.
,
McCann
,
J.
,
Pollard
,
N.
, and
Faloutsos
,
C.
,
2010
, “
BoLeRO: A Principled Technique for Including Bone Length Constraints in Motion Capture Occlusion Filling
,”
Proceedings of the 2010 ACM SIGGRAPH/Eurographics Symposium on Computer Animation
, pp.
179
188
.
28.
Ho
,
E. S. L.
,
Komura
,
T.
, and
Tai
,
C.-L.
,
2010
, “
Spatial Relationship Preserving Character Motion Adaptation
,”
ACM Trans. Graph.
,
29
(
4
), pp.
33:1
33:8
.10.1145/1778765.1778770
29.
Caon
,
M.
,
Yue
,
Y.
,
Tscherrig
,
J.
,
Mugellini
,
E.
, and
Khaled
,
O. A.
,
2011
, “
Context-Aware 3D Gesture Interaction Based on Multiple Kinects
,”
Proceedings of The First International Conference on Ambient Computing
, Applications, Services and Technologies, AMBIENT 2011.
30.
Holte
,
M. B.
,
Tran
,
C.
,
Trivedi
,
M. M.
, and
Moeslund
,
T. B.
,
2012
, “
Human Pose Estimation and Activity Recognition from Multi-View Videos: Comparative Explorations of Recent Developments
,”
IEEE J. Sel. Top. Signal Process.
,
6
(
5
), pp.
538
552
.10.1109/JSTSP.2012.2196975
31.
Rusinkiewicz
,
S.
,
Brown
,
B.
, and
Kazhdan
,
M.
, “
3D Scan Matching and Registration. ICCV 2005 Short Course
,” available at: http://www.cs.princeton. edu/bjbrown/iccv05_course
32.
Shoemake
,
K.
,
1985
, “
Animating Rotation With Quaternion Curves
,”
SIGGRAPH Comput. Graph.
,
19
, pp.
245
254
.10.1145/325165.325242
33.
Maimone
,
A.
, and
Fuchs
,
H.
,
2012
, “
Reducing Interference Between Multiple Structured Light Depth Sensors Using Motion
,”
2012 IEEE Virtual Reality Conference
, pp.
51
54
.
34.
Plagemann
,
C.
,
Ganapathi
,
V.
,
Koller
,
D.
, and
Thrun
,
S.
,
2010
, “
Real-Time Identification and Localization of Body Parts From Depth Images
,”
2010 IEEE International Conference on Robotics and Automation
, pp.
3108
3113
.
35.
Kalogerakis
,
E.
,
Hertzmann
,
A.
, and
Singh
,
K.
,
2010
, “
Learning 3D Mesh Segmentation and Labeling
,”
ACM Trans. Graph.
,
29
(
4
), pp.
102:1
102:12
.10.1145/1778765.1778839
36.
Baak
,
A.
,
Müller
,
M.
,
Bharaj
,
G.
,
Seidel
,
H.-P.
, and
Theobalt
,
C.
,
2011
, “
A Data-Driven Approach for Real-Time Full Body Pose Reconstruction From a Depth Camera
,”
IEEE 13th International Conference on Computer Vision (ICCV)
, pp.
1092
1099
.
You do not currently have access to this content.