Safe Online Integral Reinforcement Learning for Control Systems via Controller Decomposition

Integral Reinforcement learning (IRL) can optimize control systems through a stable training process without analytical models. However, when online machine learning is applied to industry, the safety constraints of physical processes cannot be violated. To address the safety problem, this paper proposes a safe online IRL scheme for control systems under the control output saturation via controller decomposition. The proposed scheme decomposes a controller into several sub-controllers in which the saturation range is divided into small portions to constrain the learning error of individual sub-controllers, thereby confining the system state within the safety region. To reduce conservatism, a training procedure is proposed for the scheme, which trains the sub-controllers one by one in a loop until all of them are optimal. Theoretical analysis results indicate the proposed scheme can ensure safety under the suggested division of the controller without any performance degradation. The simulation of two cases verifies the safety of the proposed scheme under random initial parameters for learning. Additionally, the proposed scheme is compared with several existing safe reinforcement learning methods, and the results illustrate the advantages of the proposed scheme in ensuring the safety and mitigating the fluctuation of state curves during learning.