Dynamic event-triggered control for discrete-time nonlinear Markov jump systems using policy iteration-based adaptive dynamic programming. (English) Zbl 1520.93324
Summary: This paper investigates a dynamic event-triggered optimal control problem of discrete-time (DT) nonlinear Markov jump systems (MJSs) via exploring policy iteration (PI) adaptive dynamic programming (ADP) algorithms. The performance index function (PIF) defined in each subsystem is updated by utilizing an online PI algorithm, and the corresponding control policy is derived via solving the optimal PIF. Then, we adopt neural network (NN) techniques, including an actor network and a critic network, to estimate the iterative PIF and control policy. Moreover, the designed dynamic event-triggered mechanism (DETM) is employed to avoid wasting additional resources when the estimated iterative control policy is updated. Finally, based on the Lyapunov difference method, it is proved that the system stability and the convergence of all signals can be guaranteed under the developed control scheme. A simulation example for DT nonlinear MJSs with two system modes is presented to demonstrate the feasibility of the control design scheme.
MSC:
| 93C65 | Discrete event control/observation systems |
| 93C55 | Discrete-time control/observation systems |
| 93C10 | Nonlinear systems in control theory |
| 93E03 | Stochastic systems in control theory (general) |
| 93C40 | Adaptive control/observation systems |
| 90C39 | Dynamic programming |
Keywords:
Markov jump systems; neural network; dynamic event-triggered mechanism; policy iteration; adaptive dynamic programmingReferences:
| [1] | Walker, B. K., Evaluating performance and reliability of automatically reconfigurable aerospace systems using Markov modeling techniques, (Reliability and Safety Assessment of Dynamic Process Systems (1994), Springer), 101-126 |
| [2] | Wu, H.-N.; Cai, K.-Y., Mode-independent robust stabilization for uncertain Markovian jump nonlinear systems via fuzzy control, IEEE Trans. Syst. Man Cybern. B, 36, 3, 509-519 (2006) |
| [3] | Yin, Y.; Shi, P.; Liu, F.; Teo, K. L.; Lim, C.-C., Robust filtering for nonlinear nonhomogeneous Markov jump systems by fuzzy approximation approach, IEEE Trans. Cybern., 45, 9, 1706-1716 (2014) |
| [4] | Zhang, L.; Zong, G.; Zhao, X.; Zhao, N., Output reachable set synthesis of event-triggered control for singular Markov jump systems under multiple cyber-attacks, IEEE/ACM Trans. Netw., 1-9 (2022) |
| [5] | Zhang, L.; Ning, Z.; Shi, P., Inputoutput approach to control for fuzzy Markov jump systems with time-varying delays and uncertain packet dropout rate, IEEE Trans. Cybern., 45, 11, 2449-2460 (2014) |
| [6] | Tao, J.; Lu, R.; Shi, P.; Su, H.; Wu, Z.-G., Dissipativity-based reliable control for fuzzy Markov jump systems with actuator faults, IEEE Trans. Cybern., 47, 9, 2377-2388 (2016) |
| [7] | Zhao, D.; Karimi, H. R.; Sakthivel, R.; Li, Y., Non-fragile fault-tolerant control for nonlinear Markovian jump systems with intermittent actuator fault, Nonlinear Anal. Hybrid Syst., 32, 337-350 (2019) · Zbl 1425.93264 |
| [8] | Zhang, L.; Zhao, X.; Zhao, N., Real-time reachable set control for neutral singular Markov jump systems with mixed delays, IEEE Trans. Circuits Syst. II, 69, 3, 1367-1371 (2022) |
| [9] | Feng, Z.; Shi, P., Sliding mode control of singular stochastic Markov jump systems, IEEE Trans. Automat. Control, 62, 8, 4266-4273 (2017) · Zbl 1373.93305 |
| [10] | Zhang, H.; Wang, H.; Niu, B.; Zhang, L.; Ahmad, A. M., Sliding-mode surface-based adaptive actor-critic optimal control for switched nonlinear systems with average dwell time, Inform. Sci., 580, 756-774 (2021) · Zbl 07786227 |
| [11] | Gao, Y.; Liu, Y.-J., Adaptive fuzzy optimal control using direct heuristic dynamic programming for chaotic discrete-time system, J. Vib. Control, 22, 2, 595-603 (2016) |
| [12] | Mu, C.; Ni, Z.; Sun, C.; He, H., Air-breathing hypersonic vehicle tracking control based on adaptive dynamic programming, IEEE Trans. Neural Netw. Learn. Syst., 28, 3, 584-598 (2016) |
| [13] | Wang, B.; Zhao, D.; Alippi, C.; Liu, D., Dual heuristic dynamic programming for nonlinear discrete-time uncertain systems with state delay, Neurocomputing, 134, 222-229 (2014) |
| [14] | Wang, D.; Liu, D.; Zhao, D.; Huang, Y.; Zhang, D., A neural-network-based iterative GDHP approach for solving a class of nonlinear optimal control problems with control constraints, Neural Comput. Appl., 22, 2, 219-227 (2013) |
| [15] | Sokolov, Y.; Kozma, R.; Werbos, L. D.; Werbos, P. J., Complete stability analysis of a heuristic approximate dynamic programming control design, Automatica, 59, 9-18 (2015) · Zbl 1338.90442 |
| [16] | Zhao, Y.; Wang, H.; Xu, N.; Zong, G.; Zhao, X., Reinforcement learning-based decentralized fault tolerant control for constrained interconnected nonlinear systems, Chaos Solitons Fractals, 167, Article 113034 pp. (2023) |
| [17] | Wei, Q.; Lewis, F. L.; Liu, D.; Song, R.; Lin, H., Discrete-time local value iteration adaptive dynamic programming: Convergence analysis, IEEE Trans. Syst. Man Cybern. A, 48, 6, 875-891 (2016) |
| [18] | Zhang, H.; Zhao, X.; Wang, H.; Zong, G.; Xu, N., Hierarchical sliding-mode surface-based adaptive actor-critic optimal control for switched nonlinear systems with unknown perturbation, IEEE Trans. Neural Netw. Learn. Syst., 1-12 (2022) |
| [19] | Peng, Z.; Zhao, Y.; Hu, J.; Ghosh, B. K., Data-driven optimal tracking control of discrete-time multi-agent systems with two-stage policy iteration algorithm, Inform. Sci., 481, 189-202 (2019) · Zbl 1451.93228 |
| [20] | Wei, Q.; Liu, D.; Lin, Q.; Song, R., Discrete-time optimal control via local policy iteration adaptive dynamic programming, IEEE Trans. Cybern., 47, 10, 3367-3379 (2016) |
| [21] | Lin, Q.; Wei, Q.; Liu, D., A novel optimal tracking control scheme for a class of discrete-time nonlinear systems using generalised policy iteration adaptive dynamic programming algorithm, Internat. J. Systems Sci., 48, 3, 525-534 (2017) · Zbl 1358.93076 |
| [22] | Zhong, X.; He, H.; Zhang, H.; Wang, Z., Optimal control for unknown discrete-time nonlinear Markov jump systems using adaptive dynamic programming, IEEE Trans. Neural Netw. Learn. Syst., 25, 12, 2141-2155 (2014) |
| [23] | Jiang, H.; Zhang, H.; Luo, Y.; Wang, J., Optimal tracking control for completely unknown nonlinear discrete-time Markov jump systems using data-based reinforcement learning method, Neurocomputing, 194, 176-182 (2016) |
| [24] | Li, Y.; Wang, H.; Zhao, X.; Xu, N., Event-triggered adaptive tracking control for uncertain fractional-order nonstrict-feedback nonlinear systems via command filtering, Int. J. Robust Nonlinear Control, 32, 14, 7987-8011 (2022) · Zbl 1528.93139 |
| [25] | Tang, F.; Niu, B.; Zong, G.; Zhao, X.; Xu, N., Periodic event-triggered adaptive tracking control design for nonlinear discrete-time systems via reinforcement learning, Neural Netw., 154, 43-55 (2022) · Zbl 1525.93256 |
| [26] | Cao, Z.; Niu, B.; Zong, G.; Xu, N., Small-gain technique-based adaptive output constrained control design of switched networked nonlinear systems via event-triggered communications, Nonlinear Anal. Hybrid Syst., 47, 101299 (2023) · Zbl 1505.93122 |
| [27] | Zhao, Y.; Tang, F.; Zong, G.; Zhao, X.; Xu, N., Event-based adaptive containment control for nonlinear multiagent systems with periodic disturbances, IEEE Trans. Circuits Syst. II, 69, 12, 5049-5053 (2022) |
| [28] | Cao, Y.; Zhao, N.; Xu, N.; Zhao, X.; Alsaadi, F. E., Minimal-approximation-based adaptive event-triggered control of switched nonlinear systems with unknown control direction, Electronics, 11, 20, 3386 (2022) |
| [29] | Li, Y.; Niu, B.; Zong, G.; Zhao, J.; Zhao, X., Command filter-based adaptive neural finite-time control for stochastic nonlinear systems with time-varying full state constraints and asymmetric input saturation, Internat. J. Systems Sci, 53, 1, 199-221 (2022) · Zbl 1483.93553 |
| [30] | Zhang, P.; Liu, T.; Jiang, Z.-P., Input-to-state stabilization of nonlinear discrete-time systems with event-triggered controllers, Systems Control Lett., 103, 16-22 (2017) · Zbl 1370.93235 |
| [31] | Li, Q.; Shen, B.; Liu, Y.; Alsaadi, F. E., Event-triggered \(H_\infty\) state estimation for discrete-time stochastic genetic regulatory networks with Markovian jumping parameters and time-varying delays, Neurocomputing, 174, 912-920 (2016) |
| [32] | Rakkiyappan, R.; Maheswari, K.; Velmurugan, G.; Park, J. H., Event-triggered \(H_\infty\) state estimation for semi-Markov jumping discrete-time neural networks with quantization, Neural Netw., 105, 236-248 (2018) · Zbl 1441.93304 |
| [33] | Shen, H.; Su, L.; Wu, Z.-G.; Park, J. H., Reliable dissipative control for Markov jump systems using an event-triggered sampling information scheme, Nonlinear Anal. Hybrid Syst., 25, 41-59 (2017) · Zbl 1377.93105 |
| [34] | Wang, H.; Zhang, D.; Lu, R., Event-triggered \(H_\infty\) filter design for Markovian jump systems with quantization, Nonlinear Anal. Hybrid Syst., 28, 23-41 (2018) · Zbl 1380.93261 |
| [35] | Wu, X.; Mu, X., Event-triggered control for networked nonlinear semi-Markovian jump systems with randomly occurring uncertainties and transmission delay, Inform. Sci., 487, 84-96 (2019) · Zbl 1454.93168 |
| [36] | Li, M.; Liu, M.; Zhang, Y., Asynchronous adaptive quantized feedback sliding mode control for semi-Markovian jump systems: an event-triggered approach, Nonlinear Anal. Hybrid Syst., 36, Article 100853 pp. (2020) · Zbl 1441.93048 |
| [37] | Ge, X.; Han, Q.-L.; Wang, Z., A dynamic event-triggered transmission scheme for distributed set-membership estimation over wireless sensor networks, IEEE Trans. Cybern., 49, 1, 171-183 (2017) |
| [38] | Girard, A., Dynamic triggering mechanisms for event-triggered control, IEEE Trans. Automat. Control, 60, 7, 1992-1997 (2014) · Zbl 1360.93423 |
| [39] | Cai, J.; Wen, C.; Xing, L.; Qiuzhen, Y., Decentralized backstepping control for interconnected systems with non-triangular structural uncertainties, IEEE Trans. Automat. Control, 1-12 (2022) |
| [40] | Long, L.; Wang, J., Safety-critical dynamic event-triggered control of nonlinear systems, Syst. Control Lett., 162, Article 105176 pp. (2022) · Zbl 1485.93353 |
| [41] | Yao, W.; Wang, C.; Sun, Y.; Zhou, C.; Lin, H., Synchronization of inertial memristive neural networks with time-varying delays via static or dynamic event-triggered control, Neurocomputing, 404, 367-380 (2020) |
| [42] | Tian, E.; Wang, Z.; Zou, L.; Yue, D., Probabilistic-constrained filtering for a class of nonlinear systems with improved static event-triggered communication, Internat. J. Robust Nonlinear Control, 29, 5, 1484-1498 (2019) · Zbl 1410.93130 |
| [43] | Huong, D. C.; Trinh, H., On static and dynamic triggered mechanisms for event-triggered control of uncertain systems, Circuits Systems Signal Process., 39, 10, 5020-5038 (2020) · Zbl 1485.93347 |
| [44] | Zhai, J.; Niu, T.; Ye, J.; Feng, E., Optimal control of nonlinear switched system with mixed constraints and its parallel optimization algorithm, Nonlinear Anal. Hybrid Syst., 25, 21-40 (2017) · Zbl 1376.49031 |
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. In some cases that data have been complemented/enhanced by data from zbMATH Open. This attempts to reflect the references listed in the original paper as accurately as possible without claiming completeness or a perfect matching.
