Entropy-controlled intrinsic-motivation reinforcement learning for quadruped robot locomotion in complex terrains

Abstract

Learning is the basis of both biological and artificial systems when it comes to mimicking intelligent behaviors. From the classical PPO (proximal policy optimization), there is a series of deep reinforcement learning algorithms which are widely used in training locomotion policies for quadrupedal robots because of their stability and sample efficiency. In this paper, we introduce entropy-controlled intrinsic-motivation (ECIM), an entropy-based reinforcement learning algorithm in contrast with the PPO series, that can reduce premature convergence by combining intrinsic motivation with adaptive exploration. For experiments, we use the Isaac Gym to implement our algorithms across six terrain categories: upward slopes, downward slopes, uneven rough terrain, ascending stairs, descending stairs, and flat ground as widely used. For a comparison with PPO, our experiments consistently achieve better performance: task rewards increase by 4–12%, peak body pitch oscillation is reduced by 23–29%, joint acceleration decreases by 20–32%, and joint torque consumption declines by 11–20%. Overall, our model ECIM, by entropy control and intrinsic motivation control, achieves better results in stability across different terrains for quadrupedal locomotion, reduces energetic cost at the same time, and makes it a practical choice for complex robotic control tasks. Our approach supports autonomous gait modulation and dynamic balance maintenance in diverse, unstructured environments.