Reinforcement Learning Control of Nonlinear Dynamical Systems: A Dynamics-Oriented Scoping Review

Interdisciplinary research at the intersection of reinforcement learning and nonlinear control encompasses a wide range of dynamical systems, closed-loop objectives, and control mechanisms. However, existing reviews usually organise the literature by algorithm category or application scenario, making it difficult to explain both the behavior ultimately produced by the system and the logic through which the controller achieves its objectives. This scoping review aims to map the distribution and thematic structure of the literature on reinforcement learning-based control of nonlinear dynamical systems, from 2021 to 2026. It summarizes the desired closed-loop behaviors and control-design mechanisms and analyses their co-occurrence relationships. The Web of Science Core Collection, Scopus, IEEE Xplore, and five conference proceedings were searched, with eligible sources limited to English-language journal articles and conference papers. After deduplication, 3134 records were retained and rated for analytical relevance on an A–D scale. The A + B-rated corpus (n = 2211) was used for literature-distribution mapping and TF–IDF, truncated singular value decomposition, and Ward clustering analyses with sensitivity tests, while the A-rated core corpus (n = 1187) underwent manual dual-axis coding. The results show that value-function- and optimality-driven mechanisms occur across most closed-loop behavior categories, whereas orbital and motion-reproduction behaviours, synchronization and relational manifolds, and safety-set preservation primarily co-occur with data-driven policy learning, interaction and resource awareness, and safety, robustness, and reliability mechanisms, respectively. Overall, the reviewed literature remains dominated by regulation and tracking and exhibits a many-to-many structure in which value-function- and optimality-driven mechanisms provide a common foundation, while other mechanisms are combined according to the closed-loop objective. The main contribution of this review is a dual-axis analytical framework that links desired closed-loop behaviors to control-design mechanisms. This framework moves beyond the traditional organisation of evidence solely by algorithm category and reveals the structural interdependence between control objectives and design mechanisms. Future research should further investigate transitions between closed-loop behaviors and the integrated design of policy learning with stability, safety, resource constraints, and fault handling.