This article describes benefits of model-based approach in developing humanoids and presents experimental results as well. Regardless of the chosen actuation strategy, the design of humanoid locomotion controllers is greatly complicated by the underactuated and nonlinear nature of the associated multibody dynamics. Drawing inspiration from biology, researchers have begun to incorporate passive mechanical compliance into the design of legged robots, often by adding spring elements in series with the robot’s actuators. First introduced by the MIT Leg Laboratory, series elastic actuators (SEAs) have been shown to improve the fidelity and stability of closed-loop force controllers while simultaneously increasing shock tolerance. The chapter shows an example SEA utilized in the design of THOR, a compliant humanoid robot developed at Virginia Tech. Despite new advancements, several challenges remain before humanoids can be fielded in real-world applications that require a high degree of mobility. Model-based control approaches could greatly benefit from techniques found in the robust and adaptive control literature. The field is also interested in moving towards more efficient, human-like locomotion using biologically-inspired control strategies.

## Article

As bipeds, humanoid robots have the potential to far surpass the performance of wheeled platforms in tasks requiring a high degree of mobility in natural and man-made environments. In nature, bipeds such as humans and birds are highly adept at traversing difficult and diverse terrain with both speed and agility. In order for humanoids to reach their full utility, comparable control strategies will need to be developed for legged robots. Legged animals often employ highly dynamic gaits to achieve fast and efficient locomotion, despite limited range of motion and muscle strength. To do so, animals utilize active and passively compliant control strategies to maintain balance in response to external disturbances and unpredictable terrain [1].

It is no surprise that compliant control strategies offer a number of advantages for humanoid locomotion. Low joint stiffness can help minimize damage to mechanisms and actuators in the event of a collision and improve the safety of robots operating in close proximity to humans. In the robotics literature, compliant control is often achieved using force controllable actuators. The ability to closely regulate internal and external forces during locomotion enables naturally compliant behaviors capable of adapting to uncertain terrain features encountered in real-world environments. Regardless of the chosen actuation strategy, the design of humanoid locomotion controllers is greatly complicated by the underactuated and nonlinear nature of the associated multibody dynamics. Additional difficulties arise due to the significant unmodeled dynamics present in real mechanical systems.

High feedback gains can help to mitigate the effects of unmodeled dynamics during locomotion at the expense of “stiff” and often unpredictable interactions with the environment. On the other hand, compliant locomotion strategies permit excellent control of interaction forces, yet require accurate feedforward planning and control to achieve comparable tracking performance. As such, one of the major challenges in developing compliant locomotion strategies is selecting appropriate dynamic models. Simple models are commonly employed to aid in the design of robust, real-time control laws, while complex models offer increased dynamic accuracy for torque-controlled humanoids. Given a suitable model, the remaining challenge lies in developing high-performance, low-stiffness controllers that can adapt to a wide variety of challenging and unpredictable environments.

## Compliant Actuation

Drawing inspiration from biology, researchers have begun to incorporate passive mechanical compliance into the design of legged robots, often by adding spring elements in series with the robot's actuators. First introduced by the MIT Leg Laboratory, series elastic actuators (SEAs) have been shown to improve the fidelity and stability of closed-loop force controllers while simultaneously increasing shock tolerance [2]. Figure 1 shows an example SEA utilized in the design of THOR, a compliant humanoid robot developed at Virginia Tech. Linear forces are generated by a brushless DC motor that drives a two-stage transmission composed of a belt drive and precision ball screw. The actuator's spring element consists of a compliant titanium beam subjected to moment loading.

### Series Elastic Actuator Model

SEAs are often modeled as simple spring-mass-damper systems assuming a fixed output load. The open-loop transfer function is given by
$FI=γ∙km∙s2+b∙s+k$
(1)
where F is the output force, I is the commanded motor current, γ is the D.C. gain, k is the spring stiffness, b is the transmission damping, and m is the equivalent lumped mass of the motor, transmission, and reflected rotor inertia [3]. The spring stiffness selection is a key design variable for SEAs. When tracking a zero force reference, the spring deflects immediately under load, allowing the motor additional time to regulate the output force by accelerating the transmission. As such, a design tradeoff exists due to the favorable low mechanical impedance of a soft spring and high control bandwidth of a stiff spring.

### Tracking Force and Torque Trajectories

Force and torque control strategies for linear and rotary SEAs typically rely on some variation of feedforward and feedback control to accelerate the rotor in response to the estimated spring deflection [2, 4]. The THOR SEA force controller combines feedforward and PID control with a model-based disturbance observer (DOB) based on measurements from an inline load cell sensor. The DOB estimates unmodeled disturbances such as nonlinear friction forces and attempts to cancel their effect through feedforward control. Given the measured output force, the disturbance signal is estimated by comparing the commanded motor current with the estimated motor current obtained from an inverse model of the open-loop actuator plant in Eq. (1). The combined approach offers excellent force tracking with a closed-loop bandwidth of 30 Hz at 200 N (approximately 10% of the peak actuator force), despite observing roughly 200 N of stiction in the actuator transmission.

The THOR humanoid features a rigid skeletal structure with rotary joints similar to the human musculoskeletal system. Shown in Figure 3, the robot's linear SEAs are arranged in serial and parallel configurations spanning the hip, knee, and ankle joints. For the purposes of high-level control, it is often convenient to command the torques and accelerations about each joint, as opposed to the individual forces imparted by each actuator. A simple solution is to track equivalent actuator forces obtained from an inverse statics solution for each joint mechanism. Figure 2 shows the desired and estimated joint torque trajectories for the hip pitch joint using the aforementioned approach. For this experiment, the torque reference trajectory was selected to emulate the motion of the swing leg during walking. The results show that the use of a disturbance observer significantly reduces torque tracking errors due to stiction and model uncertainty.

## Whole-Body Control

The underactuated mechanics of legged locomotion have inspired decades of diverse research into control techniques for humanoid robots. These techniques often rely on position and velocity feedback obtained from proprioception sensors to compute admissible joint setpoints for whole-body control. In order to reduce the complexity of the high-dimensional control problem, whole-body behaviors such as dynamic walking are often decomposed into constituent motion tasks that can be tracked using simpler low-dimensional controllers. Example tasks include positioning the swing foot, rotating the upper body, and accelerating the center of mass. Many torque-based whole-body control strategies employ convex optimization techniques to minimize the aggregate error associated with each motion task [5–7. In order to compute optimal torque setpoints for each joint, these techniques require an approximate model of the whole-body dynamics.

### Rigid Body Humanoid Model

While mechanical compliance can dramatically improve the performance of low-level force and torque control strategies, structural rigidity is often highly valued in the design of legged robots. Assuming the deformation of each link is sufficiently small under load, articulated humanoids are typically modeled as rigid body systems. In order to decouple the actuator dynamics from the high-level controller design, a number of approaches assume that the actuator controller is capable of approximating an ideal torque source at each joint [4–7. In this case, the rigid-body equations of motion are given by
$0τ=Hqq̈+Cq,q˙−∑IJITfI$
(2)
where $τ∈Rn$ is the vector of joint torques, $q∈Rn+6$ is the vector of generalized coordinates (including the joint positions and 6 DOF floating base frame), H(q) is the joint-space inertia matrix, and $Cq,q˙$ is the vector of centrifugal, Coriolis and gravity torques. The final term accounts for external contact forces acting on the robot, where Ji and fi represent the point Jacobians and force vectors corresponding to each contact point, i=1...N.

### Solving The Inverse Dynamics

Whole-body control of the THOR humanoid is achieved using an inverse dynamics solver that computes admissible joint torques given a set of desired motion tasks including Cartesian accelerations, joint accelerations, and momentum rates of change [7]. Task-space accelerations and forces can, in general, be expressed as a linear combination of the joint velocities and accelerations. This allows the inverse dynamics problem to be formulated as an efficient quadratic program (QP). The QP is designed to minimize a cost function based on the weighted sum of squared errors for each motion task, where the decision variables include the joint accelerations and generalized contact forces. The relative weight of each cost term can be tuned to enforce a soft prioritization of motion tasks. To ensure that the optimized values are achievable by the hardware platform, the optimization includes constraints related to the available control authority, range of motion, and frictional contact points.

Given the optimized accelerations and contact forces, the corresponding joint torque setpoints are computed from the rigid body equations of motion given by Eq. (2). Due to model uncertainty, instability issues sometimes arise when implementing inverse dynamics approaches on torque-controlled hardware platforms. Low-level damping can help improve joint stability at the expense of dynamic accuracy. To enable whole-body control of the THOR humanoid, low-gain velocity feedback is introduced into the actuator force controller using estimates obtained from pre-transmission motor encoders. The actuator velocity setpoints are obtained by integrating the optimized joint accelerations and solving the forward velocity kinematics for each joint mechanism.

## Compliant Locomotion

Dynamic stability, accurate foot placement, and low cost of transport are qualities of human walking that are often desired of humanoid locomotion. Unfortunately, the underlying control policies employed by humans and legged animals remain largely unknown. In the humanoid literature, momentum control has become an increasingly popular method to stabilize a robot's centroidal dynamics during locomotion [8]. Force and torque-controlled humanoids are well-suited to this approach, since momentum rates of change are related to the forces and torques acting on the system.

### Centroidal Dynamics

The centroidal dynamics of a rigid body humanoid define the reduced equations of motion for the center of mass, $x∈R3$, and linear momentum, $l∈R3$. The CoM acceleration and linear momentum rate of change are governed by Newton's second law, $mẍ=l˙=∑fi+fg$, where $∑fi$ is the total contact force and fg is the force of gravity. For dynamic planning and control purposes, it is often convenient to reason in terms of geometric reference points as opposed to the contact forces acting on the system.

The Virtual Repellent Point (VRP) and enhanced Centroidal Moment Pivot (eCMP) define the direction and magnitude of the linear momentum rate of change and total contact force in terms of the CoM position [9]. These closely related reference points can be expressed as
$i=ω02x−rvrp ∑fi=ω02x−rcmp$
(3)
where $rvrp∈R3$ is the VRP position, $rcmp∈R3$ is the eCMP position, and
$ω0=gΔzcom$
is the natural frequency of the second order CoM dynamics assuming a gravitational constant, g, and CoM height, Δz. As illustrated in Figure 4, the VRP lies directly above the eCMP at the nominal CoM height. The eCMP typically lies in the robot's base of support, i.e. the convex hull formed by the contact points on each support foot. When the robot's center of pressure (CoP) is aligned with the eCMP, the line of action for the total contact force passes through the CoM. As a result, it is generally possible to avoid generating angular momentum about the CoM assuming the eCMP does not leave the base of support.

Momentum control approaches can be further simplified by defining an appropriate linear transformation of the center of mass state. The three-dimensional Divergent Component of Motion (DCM), defined as
$ξ=x1ω0x˙$
divides the CoM dynamics into stable and unstable first-order equations of motion [9],
$x˙=ω0ξ−x ξ˙=ω0ξ−rvrp$
(4)

Intuitively, the DCM represents the point at which the VRP must be placed at any point in time to allow the CoM to come to a complete rest. The DCM diverges from the VRP with a time constant of 1/ωo, while the CoM converges to the DCM at the same rate. As a result, the centroidal dynamics can be indirectly stabilized using a simple VRP-based control law to regulate the DCM position where the commanded VRP is mapped to a corresponding linear momentum rate of change objective using Eq. (3).

### Designing a Walking Controller

The THOR walking controller is implemented using a simple state machine that responds to external events such as toe-off and heel-strike. At the beginning of each step, the controller computes dynamically feasible joint-space and task-space trajectories given desired foothold poses and step phase durations from a high-level footstep planner. Several low-dimensional controllers are defined to compute whole-body motion tasks that stabilize the open-loop gait. Spatial accelerations for the pelvis orientation and swing foot pose are computed using Cartesian PID controllers to track appropriate 6 DOF trajectories during each step phase, while joint-space accelerations are computed using PD feedback to achieve the desired upper body motion. Angular momentum rate of change objectives are computed using a simple damping controller to regulate the total angular momentum, while linear momentum rate of change objectives are computed using a PI controller to track a dynamically feasible DCM trajectory.

Each controller relies on accurate feedforward trajectories to decrease the minimum feedback gains required to achieve satisfactory tracking performance. When computing the inverse dynamics, relatively high weights are assigned to the linear momentum and swing foot acceleration objectives, while relatively low weights are assigned to the angular momentum and upper body acceleration objectives. This heuristic encourages accurate foot placement, while allowing the upper body DOF to assist the DCM controller for dynamic balance.

## Experimental Results

The THOR walking controller has been tested on a variety of terrain types. Figure 5 shows the estimated footholds and reference point trajectories while walking forward on flat terrain at an average velocity of 0.08 m/s. Note that the eCMP reference passes through the center of the robot's base of support. The VRP setpoint deviates from the reference eCMP trajectory in order to stabilize the DCM trajectory. If the VRP remains within the base of support, the whole-body optimization is generally able to avoid generating significant torque about the CoM by minimizing the distance between the CoP and VRP.

Figure 6 shows the horizontal DCM and CoM trajectories corresponding to the first five steps of Fig. 5. The double support duration is 0.7 s, the single support duration is 1.05 s, and the stride length is 0.14 m. The DCM reference is computed using a combination of reverse-time integration and model predictive control given a nominal eCMP reference trajectory over a 3-step preview window. Accurate inverse dynamics and high-performance torque control enable a high degree of compliance while tracking the DCM, resulting in an inherent robustness to unmodeled terrain features. Figure 7 shows the robot walking on gravel and grass. Note that the whole-body controller weights and gains used for these outdoor experiments are identical to those used on flat terrain.

## Next Steps

Over the past decade, the robotics community has made significant progress in the development of effective compliant locomotion strategies for force-controlled humanoids. As walking and running controllers continue to improve, we hope to gain new insights into the underlying mechanics of human locomotion. Despite new advancements, a number of challenges remain before humanoids can be fielded in real-world applications that require a high degree of mobility. Model-based control approaches could greatly benefit from techniques found in the robust and adaptive control literature. The field is also interested in moving towards more efficient, human-like locomotion using biologically-inspired control strategies.

## Acknowledgments

This material was supported by ONR through grant N00014-11-1-0074 and by DARPA through grant N65236-12-1-1002. We would like to thank everyone who contributed to the development of the THOR humanoid, including Dennis Hong, Bryce Lee, Coleman Knabe, Viktor Orekhov, Derek Lahr, Jacob Webb, Steve Ressler, and Jack Newton. This material is based on work supported by (while Alexander Leonessa serves at) the National Science Foundation.

## Reference

Reference
1.
Daley
,
M. A.
, and
Biewener
,
A. A.
,
2006
,
Running Over Rough Terrain Reveals Limb Control for Intrinsic Stability
,
Proceedings of the National Academy of Sciences
,
103
(
42
), pp.
15681
15686
2.
Pratt, G., and Williamson, M., 1995 “Series Elastic Actuators,”in Intelligent Robots and Systems (IROS), IEEE/RSJ International Conference on. ’Human Robot Interaction and Cooperative Robots, 1, pp. 399-406.
3.
Robinson, D., Pratt, J., Paluska, D., and Pratt, G., 1999 “Series Elastic Actuator Development for a Biomimetic Walking Robot,” in Advanced Intelligent Mechatronics, IEEE/ASMEInternational Conference on, pp. 561-568.
4.
Pain
,
N.
,
Mehling
,
J. S.
,
Holley
,
J.
,
,
N. A.
,
Johnson
,
G.
,
Fok
,
C.-L.
, and
Sentis
,
L.
,
2015
,
Actuator Control for the NASA-JSC Valkyrie Humanoid Robot: A Decoupled Dynamics Approach for Torque Control of Series Elastic Robots
,
Journal of Field Robotics
,
32
(
3
), pp.
378
396
5.
Feng
,
S.
,
Whitman
,
E.
,
Xinjilefu
,
X.
, and
Atkeson
,
C. G.
,
2014
,
Optimization-Based Full Body Control for the DARPA Robotics Challenge
,
Journal of Field Robotics
,
7
(
1
), pp.
1
20
6.
Herzog, A., Righetti, L., Grimminger, F., Pastor, P., and Schaal, S., 2013 “Balancing Experiments on a Torque-Controlled Humanoid with Hierarchical Inverse Dynamics,” arXiv preprint arXiv:1305.2042.
7.
Hopkins, M. A., Hong, D. W., and Leonessa, A., 2015 “Compliant Locomotion Using Whole-Body Control and Divergent Component of Motion Tracking,” Robotics and Automation (ICRA), IEEE International Conference on.
8.
Orin
,
D. E.
,
Goswami
,
A.
, and
Lee
,
S.-H.
,
2013
,
Centroidal Dynamics of a Humanoid Robot
,
Autonomous Robots
,
35
(
2-3
), pp.
1
16
9.
Englsberger, J., Ott, C., and Albu-Schaffer, A., 2013 “Three-Dimensional Bipedal Walking Control Using Divergent Component of Motion,” in Intelligent Robots and Systems (IROS), IEEE/RSJ International Conference on, pp. 2600-2607.