A new iterative algorithm for solving \(H_{\infty}\) control problem of continuous-time Markovian jumping linear systems based on online implementation. (English) Zbl 1351.93055
Summary: A new online iterative algorithm for solving the \(H_{\infty}\) control problem of continuous-time Markovian jumping linear systems is developed. For comparison, an available offline iterative algorithm for converging to the solution of the \(H_{\infty}\) control problem is firstly proposed. Based on the offline iterative algorithm and a new online decoupling technique named subsystems transformation method, a set of linear subsystems, which implementation in parallel, are obtained. By means of the adaptive dynamic programming technique, the two-player zero-sum game with the coupled game algebraic Riccati equation is solved online thereafter. The convergence of the novel policy iteration algorithm is also established. At last, simulation results have illustrated the effectiveness and applicability of these two methods.
MSC:
| 93B40 | Computational methods in systems theory (MSC2010) |
| 93B36 | \(H^\infty\)-control |
| 60J65 | Brownian motion |
| 90C39 | Dynamic programming |
| 93C40 | Adaptive control/observation systems |
| 91A05 | 2-person games |
| 93C05 | Linear systems in control theory |
| 93C15 | Control/observation systems governed by ordinary differential equations |
Keywords:
Markov jump linear systems (MJLSs); \(H_{\infty}\) control; subsystems transformation; coupled game algebraic Riccati equation (CGARE); onlineReferences:
| [1] | AI‐TamimiA, LewisFL. Discrete‐time nonlinear HJB solution using approximate dynamic programming: convergence proof. IEEE Transactions on Systems, Man, and Cybernetics, Part B: Cybernetics2008; 38(4):943-949. |
| [2] | JiangY, JiangZ. Computational adaptive optimal control for continuous‐time linear systems with completely unknown dynamic. Automatica2012; 48(10):2699-2704. · Zbl 1271.93088 |
| [3] | MurrayJJ, CoxCJ, LendarisGG, SaeksR. Adaptive dynamic programming. IEEE Transactions on Systems, Man and Cybernetics2002; 32(2):140-153. |
| [4] | VamvoudakisKG, LewisFL. Online actor‐critic algorithm to solve the continuous‐time infinite horizon optimal control problem. Automatica2010; 46(5):878-888. · Zbl 1191.49038 |
| [5] | VrabieD, PastravanuO, Abu‐KhalafM, LewisFL. Adaptive optimal control for continuous‐time linear systems based on policy iteration. Automatica2009; 45(2):477-484. · Zbl 1158.93354 |
| [6] | WangF, JinN, LiuD, WeiQ. Adaptive dynamic programming for finite‐horizon optimal control of discrete‐time nonlinear systems with ϵ‐error bound. IEEE Transactions on Neural Networks2011; 22(1):24-26. |
| [7] | VamvoudakisKG, LewisFL. Online solution of nonlinear two‐player zero‐sum games using synchronous policy iteration. International Journal of Robust and Nonlinear Control2012; 22(13):1460-1483. · Zbl 1292.91011 |
| [8] | VrabieD, LewisFL. Adaptive dynamic programming for online solution of a zero‐sum differential game. Journal of Control Theory and Applications2011; 9(3):353-360. · Zbl 1249.90308 |
| [9] | WuH, LuoB. Simultaneous policy update algorithms for learning the solution of linear continuous‐time H_∞ state feedback control. Information Sciences2013; 222: 472-485. · Zbl 1293.93275 |
| [10] | JiYD, ChizeckHJ. Controllability, stabilizability, and continuous‐time Markovian jump linear quadratic control. IEEE Transactions on Automatic Control1990; 35(7):777-788. · Zbl 0714.93060 |
| [11] | BoukasEK. Stochastic Switching Systems: Analysis and Design. Basel, Berlin: Birkhauser, 2005. |
| [12] | CostaOLV, do ValJBR, GeromelJC. Continuous‐time state‐feedback H_2‐control of Markovian jump linear systems via convex analysis. Automatica1999; 35(2):259-268. · Zbl 0939.93041 |
| [13] | HeS, LiuF. Finite‐time H_∞ fuzzy control of nonlinear jump systmes with time delays via dynamic observer‐based state feedback. IEEE Transactions on Fuzzy systems2012; 20(4):605-614. |
| [14] | RamiMA, GhaouiEEl. Robust state‐feedback stabilization of jump linear systems via LMIs. International Journal of Robust Nonlinear Control1996; 6(910):1015-1022. · Zbl 0863.93067 |
| [15] | DraganV, IvanovI. A numerical procedure to compute the stabilising solution of game theoretic Riccati equations of stochastic control. International Journal of Control2011; 84(4):783-800. · Zbl 1245.93136 |
| [16] | GajicZ, BornoI. Lyapunov iterations for optimal control of jump linear systems at steady state. IEEE Transactions on Automatic Control1995; 40(11):1971-1975. · Zbl 0837.93073 |
| [17] | QianY, PangW. An impicit sequenctial algorithm for solving coupled Lyapunov equations of continuous‐time Markovian jump systems. Automatica2015; 60: 245-250. · Zbl 1331.93229 |
| [18] | ShengL, ZhangW, GaoM. Some remarks on infinite horizon stochastic H_2/H_∞ control with (x,u,v)‐dependent noise and Markov jumps. Journal of the Franklin Institute2015; 352(10):3929-3946. · Zbl 1395.93200 |
| [19] | BornoI. Parallel computation of the solutions of coupled algebraic Lyapunov equations. Automatica1995; 31(9):1345-1347. · Zbl 0825.93992 |
| [20] | HeS, SongJ, DingZ, LiuF. Online adaptive optimal control for continuous‐time Markov jump linear systems using a novel policy iteration algorithm. IET Control Theory and Applications2015; 9(10):1536-1543. |
| [21] | RobinsonV, SworderD. A computational algorithm for design of regulators for linear jump parameter systems. IEEE Transactions on Automatic Control1974; 19(1):47-49. · Zbl 0276.93067 |
| [22] | FengY, AndersonBDO. An iteration algorithm to solve state‐perturbed stochastic algebraic Riccati equations in LQ zero‐sum games. Systems and Control Letters2010; 59(1):50-56. · Zbl 1186.93077 |
| [23] | MouL, YongJ. Two‐person zero‐sum linear quadratic stochastic differential games by a Hilbert space method. Journal of Industrial and Management Optimization2006; 2(1):93-115. |
| [24] | BartelsRH, StewartGW. Solution of the matrix equation AX + BX = C. Communication of ACM1972; 15(9):820-826. · Zbl 1372.65121 |
| [25] | LiZ, ZhouB, LamJ, WangY. Positive operator based iterative algorithms for solving Lyapunov equations for Itô stochastic systems with Markovian jumps. Applied Mathematics and Computation2011; 217(21):8179-8195. · Zbl 1225.93123 |
| [26] | AliprantisCD, BurkinshawO. Positive Operators. Academic Press: New York, 1985. · Zbl 0608.47039 |
| [27] | SchneiderH. Positive operators and an inertia theorem. Numerische Mathematik1965; 7(1):11-17. · Zbl 0158.28003 |
| [28] | GajicZ, LosadaR. Monotonicity of algebraic Lyapunov iterations for optimal control of jump parameter linear systems. System and Control Letters2000; 41(3):175-181. · Zbl 0985.93017 |
| [29] | KantorovichL, AkilovG. Functional Analysis in Normed Spaces. Macmillan: New York, 1964. |
| [30] | ShengL, ZhangW, GaoM. Relationship between Nash equilibrium strategies and H_2/H_∞ control of stochastic Markov jump systems with multiplicative noise. IEEE Transactions on Automatic Control2014; 59(9):2592-2597. · Zbl 1360.93786 |
| [31] | LuoB, WuH. Computationally efficient simultaneous policy update algorithm for nonlinear H_∞ state feedback control with Galerkin’s method. International Journal of Robust and Nonlinear Control2013; 23(9):991-1012. · Zbl 1270.93044 |
| [32] | BradtkeSJ, YdstieBE, BartoAG. Adaptive linear quadratic control using policy iteration. Proceedings of the American Control Conference1994; 3: 3475-3479. |
| [33] | HuangH, LongF, LiC. Stabilization for a class of Markovian jump linear systems with linear fractional uncertainties. International Journal of Innovative Computing, Information and Control2015; 11(1):295-307. |
| [34] | ShiP, LiF. A survey on Markovian jump systems: modeling and design. International Journal of Control, Automation, and Systems2015; 13(1):1-16. |
| [35] | LiF, ShiP, WuL, BasinMV, LimCC. Quantized control design for cognitive radio networks modeled as nonlinear semi‐Markovian jump systems. IEEE Transactions on Industrial Electronics2015; 62(4):2330-2340. |
| [36] | LiF, WuL, ShiP, LimCC. State estimation and sliding mode control for semi‐Markovian jump systems with mismatched uncertainties. Automatica2015; 51: 385-393. · Zbl 1309.93157 |
| [37] | ZhongX, HeH, ZhangH, WangZ. Optimal control for unknown discrete‐time nonlinear Markov jump systems using adaptive dynamic programming. IEEE Transactions on Neural Networks and Learning Systems2014; 25(12):2141-2155. |
| [38] | WeiQ, LiuD, LinH. Value iteration adaptive dynamic programming for optimal control of discrete‐time nonlinear systems. IEEE Transactions on Cybernetics2015. 10.1109/TCYB.2015.2492242. |
| [39] | WangZ, NiuY, ShenB. Analysis and control with randomly occurring incomplete information. International Journal of Systems Science2014; 45(7):1333-1336. · Zbl 1298.00192 |
| [40] | WuZ, ShiP, SuH, ChuJ. Asynchronous L_2‐L_∞ filtering for discrete‐time stochastic Markov jump systems with randomly occurred sensor nonlinearities. Automatica2014; 50(1):180-186. · Zbl 1417.93317 |
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. In some cases that data have been complemented/enhanced by data from zbMATH Open. This attempts to reflect the references listed in the original paper as accurately as possible without claiming completeness or a perfect matching.
