Robotic leg prostheses and exoskeletons have traditionally been designed using highly-geared motor-transmission systems that minimally exploit the passive dynamics of human locomotion, resulting in inefficient actuators that require significant energy consumption and thus provide limited battery-powered operation or require large onboard batteries. Here we review two of the leading energy-efficient actuator design principles for legged and wearable robotic systems: series elasticity and backdrivability. As shown by inverse dynamic simulations of walking, there are periods of negative joint work that can increase efficiency by recycling some of the otherwise dissipated energy using series elastic actuators and/or backdriveable actuators with energy regeneration. Series elastic actuators can improve shock tolerance during foot-ground impacts and reduce the peak power and energy consumption of the motor via mechanical energy storage and return. However, actuators with series elasticity tend to have lower output torque, increased mass and architecture complexity due to the added physical spring, and limited force and torque control bandwidth. High torque density motors with low-ratio transmissions, known as quasi-direct drives, can likewise achieve low output impedance and high backdrivability, allowing for safe and compliant human-robot physical interactions, in addition to energy regeneration. However, torque-dense motors tend to have higher Joule heating losses, greater motor mass and inertia, and require specialized motor drivers for real-time control. While each actuator design has advantages and drawbacks, designers should consider the energy-efficiency of robotic leg prostheses and exoskeletons during various locomotor activities of daily living.