Learning Energy Efficient Trotting For Legged Robots
Quadrupedal locomotion skills are challenging to develop. In recent years, Deep Reinforcement Learning (DRL) promises to automate the development of locomotion controllers and map sensory observations to low-level actions. However, legged locomotion still is a challenging task for DRL algorithms, especially when energy efficiency is taken into consideration. In this paper, we propose a DRL scheme for efficient trotting applied on Laelaps II quadruped in MuJoCo. First, an accurate model of the robot is created by revealing the necessary parameters to be imported in the simulation, while special focus is given to the quadruped’s drivetrain. Concerning, the reward function and the action space, we investigate the best way to integrate in the reward, the terms necessary to minimize the Cost of Transport (CoT) while maintaining a trotting locomotion pattern. Last, we present how our solution increased the energy efficiency for a simple task of trotting on level terrain similar to the treadmill-robot environment at the Control Systems Lab of NTUA.
- Mastrogeorgiou, A., Papatheodorou, A., Koutsoukis, K., and Papadopoulos, E., “Learning energy-efficient trotting for legged robots,” Robotics in Natural Settings, Lecture Notes in Networks and Systems, v. 530, Springer, 2022, pp. 204-215.
- Mastrogeorgiou, A., Papatheodorou, A., Koutsoukis, K., and Papadopoulos, E., “Learning energy-efficient trotting for legged robots,” CLAWAR 2022, Ponta Delgada, Portugal, September 12-14 2022.
Evaluating DRL Algorithms for Quadrupedal Slope Handling
In recent years, a number of deep reinforcement learning (DRL) algorithms have emerged that promise to automate the development of locomotion controllers and map sensory observations to low-level actions. However, legged locomotion still is a challenging task for DRL algorithms, especially when slope handling is required. As a result, a framework using commonly used tools (ROS, Gazebo, etc.) and specific slope handling scenarios would enable the evaluation of recent DRL algorithms in order to choose the appropriate algorithm for a given task. In this work, an evaluation framework is proposed that combines DRL with trajectory planning at toe level aiming at reducing training time and facilitate decision-making in slopehandling cases. The proposed evaluation scheme is extensively tested in a Gazebo environment and valuable results are produced using three state-of-the-art DRL algorithms.
- Mastrogeorgiou, A., Elbahrawy, Y., Machairas, K., Papadopoulos, E., “Evaluating deep reinforcement learning algorithms for quadrupedal slope handling, ” Proc. 23rd International Conference on Climbing and Walking Robots and the Support Technologies for Mobile Machines, (CLAWAR ’20), 24 – 26 August 2020, Moscow, Russia. Second Prize Award.
Slope Handling For Quadruped Robots Using DRL and Toe Trajectory Planning
Quadrupedal locomotion skills are challenging to develop. In recent years, deep Reinforcement Learning promises to automate the development of locomotion controllers and map sensory observations to low-level actions. Moreover, the full robot dynamics model can be exploited, but no model-based simplifications are to be made. In this work, a method for developing controllers for the Laelaps II robot is presented and applied to motions on slopes up to 15°. Combining deep reinforcement learning with trajectory planning at the toe level reduces complexity and training time. The proposed control scheme is extensively tested in a Gazebo environment similar to the treadmill-robot environment at the Control Systems Lab of NTUA. The learned policies produced promising results.
- Mastrogeorgiou, A., Elbahrawy, Y., Kecskeméthy, A., Papadopoulos, E., “Slope Handling for Quadruped Robots Using Deep Reinforcement Learning and Trajectory Planning,” IEEE/RSJ International Conference on Intelligent Robots and Systems, (IROS ’20), 25 – 29 October 2020, Las Vegas, NV, USA.