|Online Actor-critic Reinforcement Learning Control for Uncertain Surface Vessel Systems with External Disturbances
Van Tu Vu, Quang Huy Tran, Thanh Loc Pham, and Phuong Nam Dao*
International Journal of Control, Automation, and Systems, vol. 20, no. 3, pp.1029-1040, 2022
Abstract : This article addresses a trajectory tracking control approach for an uncertain surface vessel using the new cascade structure of adaptive reinforcement learning (ARL) algorithm and kinematic controller, feed-forward term. Since a surface vessel is decoupled by kinematic sub-system and dynamic sub-system, the cascade control system is an ideal method for obtaining the tracking problem. In the proposed control structure, the dynamic control loop is designed to be the optimized method of the corresponding dynamic sub-system and the kinematic control loop is implemented by a nonlinear controller combining with feed-forward term. The online actor-critic architecture is considered in ARL algorithm to overcome the challenge of solving the Hamilton-Jacobi-Bellman (HJB) equation. Additionally, the proposed controller is able to handle the difficulty of the non-autonomous optimal control problem by designing the ARL technique for the corresponding system with a small number of state variables. Based on theoretical analysis, the ARL based control design has been made to guarantee the uniformly ultimately bounded (UUB) stability of the closed system. Finally, the simulation results are illustrated to verify the effectiveness of the proposed control scheme.
Keyword : Actor-critic algorithm, adaptive reinforcement learning (ARL) control, cascade control systems, surface vessel (SV) systems, trajectory tracking control.