Adaptive reinforcement learning optimal tracking control for strict-feedback nonlinear systems with prescribed performance. (English) Zbl 1536.93424
Summary: The reinforcement learning-based prescribed performance optimal tracking control problem is considered for a class of strict-feedback nonlinear systems in this paper. The unknown nonlinearities and cost function are approximated by radial-basis-function (RBF) neural network (NN). The overall controller consists of an adaptive controller and an optimal compensation term. Firstly, the adaptive controller is designed by backstepping control method. Subsequently, the optimal compensation term is derived via policy iteration by minimizing cost function. In addition, depending on the prescribed performance control, the tracking error can be limited in the prescribed area. Therefore, the whole control scheme can effectively guarantee that the tracking error converges to a bound with prescribed performance while the cost function is minimized. The stability analysis shows that all signals in the closed-loop system are bounded. Finally, the effectiveness and advantages of the designed control strategy are illustrated by the simulation examples.
MSC:
| 93C40 | Adaptive control/observation systems |
| 49L20 | Dynamic programming in optimal control and differential games |
| 93B52 | Feedback control |
| 93C10 | Nonlinear systems in control theory |
| 68T05 | Learning and adaptive systems in artificial intelligence |
Keywords:
reinforcement learning; prescribed performance control; adaptive dynamic programming; tracking control; strict-feedback nonlinear systemsReferences:
| [1] | Zhang, Y.; Gao, J.; Chen, Y.; Bian, C.; Zhang, F.; Liang, Q., Adaptive neural network control for visual docking of an autonomous underwater vehicle using command filtered backstepping, Int. J. Robust Nonlinear Control, 32, 8, 4716-4738 (2022) · Zbl 1528.93121 |
| [2] | Wen, G.; Hao, W.; Feng, W.; Gao, K., Optimized backstepping tracking control using reinforcement learning for quadrotor unmanned aerial vehicle system, IEEE Trans. Syst. Man Cybern.: Syst., 52, 8, 5004-5015 (2022) |
| [3] | Ling, S.; Wang, H.; Liu, P. X., Adaptive fuzzy tracking control of flexible-joint robots based on command filtering, IEEE Trans. Industr. Electron., 67, 5, 4046-4055 (2020) |
| [4] | D. Xia, X. Yue, Y. Yin, Output-feedback asymptotic tracking control for rigid-body attitude via adaptive neural backstepping, ISA Trans. doi: 10.1016/j.isatra.2022.10.042. |
| [5] | Krstić, M.; Kanellakopoulos, I.; Kokotović, P., Adaptive nonlinear control without overparametrization, Syst. Control Lett., 19, 3, 177-185 (1992) · Zbl 0763.93043 |
| [6] | Krstic, M.; Kokotovic, P. V.; Kanellakopoulos, I., Nonlinear and adaptive control design (1995), John Wiley & Sons Inc · Zbl 0763.93043 |
| [7] | Polycarpou, M. M., Stable adaptive neural control scheme for nonlinear systems, IEEE Trans. Autom. Control, 41, 3, 447-451 (1996) · Zbl 0846.93060 |
| [8] | Kwan, C.; Lewis, F., Robust backstepping control of nonlinear systems using neural networks, IEEE Trans. Syst. Man Cybern. Part A, 30, 6, 753-766 (2000) |
| [9] | Zheng, X.; Yang, X., Command filter and universal approximator based backstepping control design for strict-feedback nonlinear systems with uncertainty, IEEE Trans. Autom. Control, 65, 3, 1310-1317 (2020) · Zbl 1533.93822 |
| [10] | Ge, S. S.; Wang, C., Direct adaptive NN control of a class of nonlinear systems, IEEE Trans. Neural Networks, 13, 1, 214-221 (2002) |
| [11] | Ge, S.; Wang, C., Adaptive neural control of uncertain MIMO nonlinear systems, IEEE Trans. Neural Networks, 15, 3, 674-692 (2004) |
| [12] | P. Parsa, M.A. T, F. Baghbani, Command-filtered backstepping robust adaptive emotional control of strict-feedback nonlinear systems with mismatched uncertainties, Inf. Sci. 579 (2021) 434-453. · Zbl 1533.93813 |
| [13] | Vafamand, N.; Arefi, M. M., Robust neural network-based backstepping landing control of quadrotor on moving platform with stochastic noise, Int. J. Robust Nonlinear Control, 32, 4, 2007-2026 (2022) · Zbl 07769340 |
| [14] | Wang, D.; Liu, D.; Li, H.; Ma, H., Neural-network-based robust optimal control design for a class of uncertain nonlinear systems via adaptive dynamic programming, Inf. Sci., 282, 167-179 (2014) · Zbl 1354.93045 |
| [15] | Li, Y.; Liu, Y.; Tong, S., Observer-based neuro-adaptive optimized control of strict-feedback nonlinear systems with state constraints, IEEE Trans. Neural Networks Learn. Syst., 33, 7, 3131-3145 (2022) |
| [16] | Yuan, L.; Li, T.; Tong, S.; Xiao, Y.; Shan, Q., Broad learning system approximation-based adaptive optimal control for unknown discrete-time nonlinear systems, IEEE Trans. Syst. Man Cybern.: Syst., 52, 8, 5028-5038 (2022) |
| [17] | Wen, G.; Niu, B., Optimized tracking control based on reinforcement learning for a class of high-order unknown nonlinear dynamic systems, Inf. Sci., 606, 368-379 (2022) · Zbl 1533.93302 |
| [18] | Chen, Y.; Wen, J.; Luan, X.; Liu, F., Optimal control for semi-markov jump linear systems via TP-free temporal difference learning, Int. J. Robust Nonlinear Control, 31, 14, 6905-6916 (2021) · Zbl 1527.93085 |
| [19] | Wen, X.; Shi, H.; Su, C.; Jiang, X.; Li, P.; Yu, J., Novel data-driven two-dimensional Q-learning for optimal tracking control of batch process with unknown dynamics, ISA Trans., 125, 10-21 (2022) |
| [20] | Chen, Z.; Chen, S.; Chen, K.; Zhang, Y., Constrained decoupling adaptive dynamic programming for a partially uncontrollable time-delayed model of energy systems, Inf. Sci., 608, 1352-1374 (2022) · Zbl 1542.90244 |
| [21] | Ma, H.; Xu, L.; Yang, G., Multiple environment integral reinforcement learning-based fault-tolerant control for affine nonlinear systems, IEEE Trans. Cybern., 51, 4, 1913-1928 (2019) |
| [22] | W. Bai, T. Li, Y. Long, C.L.P. Chen, Event-triggered multigradient recursive reinforcement learning tracking control for multiagent systems, IEEE Trans. Neural Networks Learn. Syst. doi:10.1109/TNNLS.2021.3094901. |
| [23] | Chen, Z.; Xue, W.; Li, N.; Lewis, F. L., Two-loop reinforcement learning algorithm for finite-horizon optimal control of continuous-time affine nonlinear systems, Int. J. Robust Nonlinear Control, 32, 1, 393-420 (2022) · Zbl 1527.93175 |
| [24] | T. Li, W. Bai, Q. Liu, Y. Long, C.P. Chen, Distributed fault-tolerant containment control protocols for the discrete-time multiagent systems via reinforcement learning method, IEEE Trans. Neural Networks Learn. Syst. doi:10.1109/TNNLS.2021.3121403. |
| [25] | Xue, W.; Kolaric, P.; Fan, J.; Lian, B.; Chai, T.; Lewis, F. L., Inverse reinforcement learning in tracking control based on inverse optimal control, IEEE Trans. Cybern., 52, 10, 10570-10581 (2022) |
| [26] | He, S.; Fang, H.; Zhang, M.; Liu, F.; Luan, X.; Ding, Z., Online policy iterative-based H_∞)optimization problems optimization algorithm for a class of nonlinear systems, Inf. Sci., 495, 1-13 (2019) · Zbl 1454.93060 |
| [27] | Yin, Z.; He, W.; Yang, C.; Sun, C., Control design of a marine vessel system using reinforcement learning, Neurocomputing, 311, 353-362 (2018) |
| [28] | Xue, S.; Luo, B.; Liu, D.; Gao, Y., Event-triggered ADP for tracking control of partially unknown constrained uncertain systems, IEEE Trans. Cybern., 52, 9, 9001-9012 (2022) |
| [29] | Yan, B.; Shi, P.; Lim, C.; Shi, Z., Optimal robust formation control for heterogeneous multi-agent systems based on reinforcement learning, Int. J. Robust Nonlinear Control, 32, 5, 2683-2704 (2022) · Zbl 1527.93075 |
| [30] | Deng, Y.; Liu, T.; Zhao, D., Event-triggered output-feedback adaptive tracking control of autonomous underwater vehicles using reinforcement learning, Appl. Ocean Res., 113, Article 102676 pp. (2021) |
| [31] | Guo, X.; Yan, W.; Cui, R., Integral reinforcement learning-based adaptive NN control for continuous-time nonlinear MIMO systems with unknown control directions, IEEE Trans. Syst. Man Cybern.: Syst., 50, 11, 4068-4077 (2020) |
| [32] | Zhang, H.; Wang, H.; Niu, B.; Zhang, L.; Ahmad, A. M., Sliding-mode surface-based adaptive actor-critic optimal control for switched nonlinear systems with average dwell time, Inf. Sci., 580, 756-774 (2021) · Zbl 07786227 |
| [33] | Bechlioulis, C. P.; Rovithakis, G. A., Adaptive control with guaranteed transient and steady state tracking error bounds for strict feedback systems, Automatica, 45, 2, 532-538 (2009) · Zbl 1158.93325 |
| [34] | Bechlioulis, C. P.; Rovithakis, G. A., Prescribed performance adaptive control for multi-input multi-output affine in the control nonlinear systems, IEEE Trans. Autom. Control, 55, 5, 1220-1226 (2010) · Zbl 1368.93288 |
| [35] | H. Ma, Q. Zhou, H. Li, R. Lu, Adaptive prescribed performance control of a flexible-joint robotic manipulator with dynamic uncertainties, IEEE Trans. Cybern. doi:10.1109/TCYB.2021.3091531. |
| [36] | Zhang, W.; Xu, D.; Jiang, B.; Pan, T., Prescribed performance based model-free adaptive sliding mode constrained control for a class of nonlinear systems, Inf. Sci., 544, 97-116 (2021) · Zbl 1478.93108 |
| [37] | Sui, S.; Chen, C. L.P.; Tong, S., Finite-time adaptive fuzzy prescribed performance control for high-order stochastic nonlinear systems, IEEE Trans. Fuzzy Syst., 30, 7, 2227-2240 (2022) |
| [38] | Cui, G.; Yang, W.; Yu, J.; Li, Z.; Tao, C., Fixed-time prescribed performance adaptive trajectory tracking control for a QUAV, IEEE Trans. Circuits Syst. II Express Briefs, 69, 2, 494-498 (2022) |
| [39] | Zargarzadeh, H.; Dierks, T.; Jagannathan, S., Optimal control of nonlinear continuous-time systems in strict-feedback form, IEEE Trans. Neural Networks Learn. Syst., 26, 10, 2535-2549 (2015) |
| [40] | Tong, S.; Sun, K.; Sui, S., Observer-based adaptive fuzzy decentralized optimal control design for strict-feedback nonlinear large-scale systems, IEEE Trans. Fuzzy Syst., 26, 2, 569-584 (2018) |
| [41] | Sun, K.; Li, Y.; Tong, S., Fuzzy adaptive output feedback optimal control design for strict-feedback nonlinear systems, IEEE Trans. Syst. Man Cybern.: Syst., 47, 1, 33-44 (2016) |
| [42] | Gao, X.; Bai, W.; Li, T.; Yuan, L.; Long, Y., Broad learning system-based adaptive optimal control design for dynamic positioning of marine vessels, Nonlinear Dyn., 105, 2, 1593-1609 (2021) |
| [43] | Yan, L.; Liu, Z.; Chen, C. P.; Zhang, Y.; Wu, Z., Optimized adaptive consensus control for multi-agent systems with prescribed performance, Inf. Sci., 613, 649-666 (2022) · Zbl 1536.93481 |
| [44] | Li, J.; Ji, L.; Li, H., Optimal consensus control for unknown second-order multi-agent systems: Using model-free reinforcement learning method, Appl. Math. Comput., 410, Article 126451 pp. (2021) · Zbl 1510.93020 |
| [45] | Mohammadi, M.; Arefi, M. M.; Setoodeh, P.; Kaynak, O., Optimal tracking control based on reinforcement learning value iteration algorithm for time-delayed nonlinear systems with external disturbances and input constraints, Inf. Sci., 554, 84-98 (2021) · Zbl 1485.93227 |
| [46] | Guo, Z.; Bai, W.; Zhou, Q.; Lu, R., Adaptive optimal control for a class of nonlinear systems with dead zone input and prescribed performance, Acta Autom. Sin., 45, 11, 2128-2136 (2019) · Zbl 1449.93108 |
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. In some cases that data have been complemented/enhanced by data from zbMATH Open. This attempts to reflect the references listed in the original paper as accurately as possible without claiming completeness or a perfect matching.
