Existing techniques for motion imitation often suffer a certain level of latency due to their computational overhead or a large set of correspondence samples to search. To achieve real-time imitation with small latency, we present a framework in this paper to reconstruct motion on humanoids based on sparsely sampled correspondence. The imitation problem is formulated as finding the projection of a point from the configuration space of a human's poses into the configuration space of a humanoid. An optimal projection is defined as the one that minimizes a back-projected deviation among a group of candidates, which can be determined in a very efficient way. Benefited from this formulation, effective projections can be obtained by using sparsely sampled correspondence, whose generation scheme is also introduced in this paper. Our method is evaluated by applying the human's motion captured by an RGB-depth (RGB-D) sensor to a humanoid in real time. Continuous motion can be realized and used in the example application of teleoperation.

References

References
1.
Suleiman
,
W.
,
Yoshida
,
E.
,
Kanehiro
,
F.
,
Laumond
,
J.-P.
, and
Monin
,
A.
,
2008
, “
On Human Motion Imitation by Humanoid Robot
,”
IEEE International Conference on Robotics and Automation
(
ICRA
), Pasadena, CA, May 19–23, pp.
2697
2704
.
2.
Chalodhorn
,
R.
,
Grimes
,
D. B.
,
Grochow
,
K.
, and
Rao
,
R. P.
, 2007, “
Learning to Walk Through Imitation
,” 20th International Joint Conference on Artifical Intelligence (
IJCAI
), Hyderabad, India, Jan. 6–12, Vol.
7
, pp.
2084
2090
.
3.
Nakaoka
,
S.
,
Nakazawa
,
A.
,
Kanehiro
,
F.
,
Kaneko
,
K.
,
Morisawa
,
M.
,
Hirukawa
,
H.
, and
Ikeuchi
,
K.
,
2007
, “
Learning From Observation Paradigm: Leg Task Models for Enabling a Biped Humanoid Robot to Imitate Human Dances
,”
Int. J. Rob. Res.
,
26
(
8
), pp.
829
844
.
4.
Ude
,
A.
,
Atkeson
,
C. G.
, and
Riley
,
M.
,
2004
, “
Programming Full-Body Movements for Humanoid Robots by Observation
,”
Rob. Auton. Syst.
,
47
(
2–3
), pp.
93
108
.
5.
Kim
,
S.
,
Kim
,
C.
,
You
,
B.
, and
Oh
,
S.
,
2009
, “
Stable Whole-Body Motion Generation for Humanoid Robots to Imitate Human Motions
,”
IEEE/RSJ International Conference on Intelligent Robots and Systems
(
IROS
), St. Louis, MO, Oct. 10–15, pp.
2518
2524
.
6.
Safonova
,
A.
,
Pollard
,
N. S.
, and
Hodgins
,
J. K.
,
2003
, “
Optimizing Human Motion for the Control of a Humanoid Robot
,” 2nd International Symposium on Adaptive Motion of Animals and machines (
AMAM
), Kyoto, Japan, Mar. 4–8.
7.
Ott
,
C.
,
Lee
,
D.
, and
Nakamura
,
Y.
,
2008
, “
Motion Capture Based Human Motion Recognition and Imitation by Direct Marker Control
,” Eighth
IEEE-RAS
International Conference on Humanoid Robots, Daejeon, South Korea, Dec. 1–3, pp.
399
405
.
8.
Dariush
,
B.
,
Gienger
,
M.
,
Arumbakkam
,
A.
,
Zhu
,
Y.
,
Jian
,
B.
,
Fujimura
,
K.
, and
Goerick
,
C.
,
2009
, “
Online Transfer of Human Motion to Humanoids
,”
Int. J. Humanoid Rob.
,
6
(
2
), pp.
265
289
.
9.
Do
,
M.
,
Azad
,
P.
,
Asfour
,
T.
, and
Dillmann
,
R.
,
2008
, “
Imitation of Human Motion on a Humanoid Robot Using Non-Linear Optimization
,” Eighth
IEEE-RAS
International Conference on Humanoid Robots
, Daejeon, South Korea, Dec. 1–3, pp.
545
552
.
10.
Yamane
,
K.
,
Anderson
,
S. O.
, and
Hodgins
,
J. K.
,
2010
, “
Controlling Humanoid Robots With Human Motion Data: Experimental Validation
,” Tenth
IEEE-RAS
International Conference on Humanoid Robots
, Nashville, TN, Dec. 6–8, pp.
504
510
.
11.
Koenemann
,
J.
, and
Bennewitz
,
M.
,
2012
, “
Whole-Body Imitation of Human Motions With a Nao Humanoid
,”
Seventh ACM/IEEE International Conference on Human-Robot Interaction
(
HRI
), Boston, MA, Mar. 5–8, pp.
425
425
.
12.
Koenemann
,
J.
,
Burget
,
F.
, and
Bennewitz
,
M.
,
2014
, “
Real-Time Imitation of Human Whole-Body Motions by Humanoids
,”
IEEE International Conference on Robotics and Automation
(
ICRA
), Hong Kong, China, May 31–June 7, pp.
2806
2812
.
13.
Morris
,
A. S.
, and
Mansor
,
A.
,
1997
, “
Finding the Inverse Kinematics of Manipulator Arm Using Artificial Neural Network With Lookup Table
,”
Robotica
,
26
(
6
), pp.
617
625
.
14.
Aleotti
,
J.
,
Skoglund
,
A.
, and
Duckett
,
T.
,
2004
, “
Position Teaching of a Robot Arm by Demonstration With a Wearable Input Device
,”
International Conference on Intelligent Manipulation and Grasping
(
IMG
), Genova, Italy, July 1–2.
15.
Neto
,
P.
,
Pires
,
J. N.
, and
Moreira
,
A. P.
,
2009
, “
Accelerometer-Based Control of an Industrial Robotic Arm
,”
18th IEEE International Symposium on Robot and Human Interactive Communication
(
RO-MAN
), Toyama, Japan, Sept. 27–Oct. 2, pp.
1192
1197
.
16.
Neto
,
P.
,
Pires
,
J. N.
, and
Moreira
,
A. P.
,
2010
, “
High-Level Programming and Control for Industrial Robotics: Using a Hand-Held Accelerometer-Based Input Device for Gesture and Posture Recognition
,”
Ind. Rob.: Int. J.
,
37
(
2
), pp.
137
147
.
17.
Stanton
,
C.
,
Bogdanovych
,
A.
, and
Ratanasena
,
E.
,
2012
, “
Teleoperation of a Humanoid Robot Using Full-Body Motion Capture, Example Movements, and Machine Learning
,”
Australasian Conference on Robotics and Automation
(
ACRA
), Wellington, New Zealand, Dec. 3–5, pp. 260–269.
18.
Van der Smagt
,
P.
, and
Schulten
,
K.
, 1993, “
Control of Pneumatic Robot Arm Dynamics by a Neural Network
,”
World Congress on Neural Networks
, Portland, OR, July 11–15, Vol.
3
, pp.
180
183
.
19.
Jung
,
S.
, and
Hsia
,
T.
,
1996
, “
Neural Network Reference Compensation Technique for Position Control of Robot Manipulators
,” IEEE
International Conference on Neural Networks
(
ICNN
), Washington, DC, June 3–6, Vol.
3
, pp.
1765
1770
.
20.
Larsen
,
J. C.
, and
Ferrier
,
N. J.
,
2004
, “
A Case Study in Vision Based Neural Network Training for Control of a Planar, Large Deflection, Flexible Robot Manipulator
,”
IEEE/RSJ International Conference on Intelligent Robots and Systems
(
IROS
), Sendai, Japan, Sept. 28–Oct. 2, Vol.
3
, pp.
2924
2929
.
21.
Wang
,
D.
, and
Bai
,
Y.
,
2005
, “
Improving Position Accuracy of Robot Manipulators Using Neural Networks
,”
IEEE Instrumentation and Measurement Technology Conference
(
IMTC
), Ottawa, ON, Canada, May 16–19, Vol.
2
, pp.
1524
1526
.
22.
Comaniciu
,
D.
, and
Meer
,
P.
,
2002
, “
Mean Shift: A Robust Approach Toward Feature Space Analysis
,”
IEEE Trans. Pattern Anal. Mach. Intell.
,
24
(
5
), pp.
603
619
.
23.
Huang
,
G.-B.
,
Wang
,
D. H.
, and
Lan
,
Y.
,
2011
, “
Extreme Learning Machines: A Survey
,”
Int. J. Mach. Learn. Cybern.
,
2
(
2
), pp.
107
122
.
24.
Huang
,
G.-B.
,
Zhou
,
H.
,
Ding
,
X.
, and
Zhang
,
R.
,
2012
, “
Extreme Learning Machine for Regression and Multiclass Classification
,”
IEEE Trans. Syst., Man, Cybern., Part B: Cybern.
,
26
(
2
), pp.
513
529
.
25.
Deng
,
W.
,
Zheng
,
Q.
, and
Chen
,
L.
,
2009
, “
Regularized Extreme Learning Machine
,”
IEEE Symposium on Computational Intelligence and Data Mining
(
CIDM
), Nashville, TN, Mar. 30–Apr. 2, pp.
389
395
.
26.
LaViola
,
J. J.
,
2003
, “
Double Exponential Smoothing: An Alternative to Kalman Filter-Based Predictive Tracking
,”
ACM
Workshop on Virtual Environments
, Zurich, Switzerland, May 22–23, pp.
199
206
.
27.
Lapeyre, M., Rouanet, P., Grizou, J., Nguyen, S., Depraetre, F., Le Falher, A., and Oudeyer, P.-Y., 2014, “
Poppy Project: Open-Source Fabrication of 3D Printed Humanoid Robot for Science, Education and Art
,” Digital Intelligence (
DI2014
), Nantes, France, Sept. 17–19.
28.
Yeung
,
K.-Y.
,
Kwok
,
T.-H.
, and
Wang
,
C. C.
,
2013
, “
Improved Skeleton Tracking by Duplex Kinects: A Practical Approach for Real-Time Applications
,”
ASME J. Comput. Inf. Sci. Eng.
,
13
(
4
), p.
041007
.
29.
Zheng
,
Y.
,
Chan
,
K. C.
, and
Wang
,
C. C. L.
,
2014
, “
Pedalvatar: An IMU-Based Real-Time Body Motion Capture System Using Foot Rooted Kinematic Model
,”
IEEE/RSJ International Conference on Intelligent Robots and Systems
(
IROS
), Chicago, IL, Sept. 14–18, pp.
4130
4135
.
You do not currently have access to this content.