A stochastic optimal regulator for a class of nonlinear systems. (English) Zbl 1435.93180
Summary: This work investigates an optimal control problem for a class of stochastic differential bilinear systems, affected by a persistent disturbance provided by a nonlinear stochastic exogenous system (nonlinear drift and multiplicative state noise). The optimal control problem aims at minimizing the average value of a standard quadratic-cost functional on a finite horizon. It has been supposed that neither the state of the system nor the state of the exosystem is directly measurable (incomplete information case). The approach is based on the Carleman embedding, which allows to approximate the nonlinear stochastic exosystem in the form of a bilinear system (linear drift and multiplicative noise) with respect to an extended state that includes the state Kronecker powers up to a chosen degree. This way the stochastic optimal control problem may be restated in a bilinear setting and the optimal solution is provided among all the affine transformations of the measurements. The present work is a nontrivial extension of previous work of the authors, where the Carleman approach was exploited in a framework where only additive noises had been conceived for the state and for the exosystem. Numerical simulations support theoretical results by showing the improvements in the regulator performances by increasing the order of the approximation.
MSC:
| 93E20 | Optimal stochastic control |
| 49K45 | Optimality conditions for problems involving randomness |
| 49N10 | Linear-quadratic optimal control problems |
| 93C73 | Perturbations in control/observation systems |
| 93E11 | Filtering in stochastic control theory |
References:
| [1] | Dong, L.; Wei, X.; Hu, X.; Zhang, H.; Han, J., Disturbance observer-based elegant anti-disturbance saturation control for a class of stochastic systems, International Journal of Control, 1-13 (2019) · Zbl 1454.93268 · doi:10.1080/00207179.2019.1566643 |
| [2] | Bryson, A. E.; Ho, Y. C., Applied Optimal Control (1995), New York, NY, USA: Wiley, New York, NY, USA |
| [3] | Betts, J. T., Survey of numerical methods for trajectory optimization, Journal of Guidance, Control, and Dynamics, 21, 2, 193-207 (1998) · Zbl 1158.49303 · doi:10.2514/2.4231 |
| [4] | Tang, G.-Y., Suboptimal control for nonlinear systems: a successive approximation approach, Systems and Control Letters, 54, 5, 429-434 (2005) · Zbl 1129.49303 · doi:10.1016/j.sysconle.2004.09.012 |
| [5] | Satoh, S.; Kappen, H. J.; Saeki, M., An iterative method for nonlinear stochastic optimal control based on path integrals, IEEE Transactions on Automatic Control, 62, 1, 262-276 (2017) · Zbl 1359.93541 · doi:10.1109/tac.2016.2547979 |
| [6] | Rutquist, P.; Breitholtz, C.; Wik, T., On the infinite time solution to state-constrained stochastic optimal control problems, Automatica, 44, 7, 1800-1805 (2008) · Zbl 1149.93351 · doi:10.1016/j.automatica.2007.10.018 |
| [7] | Aliyu, M. D. S., A transformation approach for solving the Hamilton-Jacobi-Bellman equation in \(H_2\) deterministic and stochastic optimal control of affine nonlinear systems, Automatica, 39, 7, 1243-1249 (2003) · Zbl 1045.93046 · doi:10.1016/s0005-1098(03)00080-3 |
| [8] | Exarchos, I.; Theodorou, E. A., Stochastic optimal control via forward and backward stochastic differential equations and importance sampling, Automatica, 87, 159-165 (2018) · Zbl 1378.93144 · doi:10.1016/j.automatica.2017.09.004 |
| [9] | Exarchos, I.; Theodorou, E. A.; Tsiotras, P., StochasticL1-optimal control via forward and backward sampling, Systems and Control Letters, 118, 101-108 (2018) · Zbl 1402.93265 · doi:10.1016/j.sysconle.2018.06.005 |
| [10] | Carravetta, F.; Mavelli, G., Suboptimal stochastic linear feedback control of linear systems with state- and control-dependent noise: the incomplete information case, Automatica, 43, 5, 751-757 (2007) · Zbl 1117.93337 · doi:10.1016/j.automatica.2006.09.010 |
| [11] | Floris, C., Numeric solution of the Fokker-Planck-Kolmogorov equation, Engineering, 5, 12, 975-988 (2013) · doi:10.4236/eng.2013.512119 |
| [12] | Bangerth, W.; Rannacher, R., Adaptive Finite Element Methods for Differential Equations (2013), Basel, Switzerland: Birkhäuser, Basel, Switzerland · Zbl 0948.65098 |
| [13] | Budhiraja, A.; Chen, L.; Lee, C., A survey of numerical methods for nonlinear filtering problems, Physica D: Nonlinear Phenomena, 230, 1-2, 27-36 (2007) · Zbl 1114.65301 · doi:10.1016/j.physd.2006.08.015 |
| [14] | Gregory, A.; Cotter, C. J.; Reich, S., Multilevel ensemble transform particle filtering, SIAM Journal on Scientific Computing, 38, 3, A1317-A1338 (2016) · Zbl 1338.65006 · doi:10.1137/15m1038232 |
| [15] | Jazwinski, A. H., Stochastic Processes and Filtering Theory (1970), Cambridge, MA, USA: Academic Press, Cambridge, MA, USA · Zbl 0203.50101 |
| [16] | Cacace, F.; Conte, F.; Germani, A.; Palombo, G., Quadratic filtering for non-Gaussian and not asymptotically stable linear discrete-time systems, Proceedings of the 53rd IEEE Conference on Decision and Control (CDC) |
| [17] | Cacace, F.; Conte, F.; Germani, A.; Palombo, G., Feedback quadratic filtering, Automatica, 82, 158-164 (2017) · Zbl 1376.93105 · doi:10.1016/j.automatica.2017.04.046 |
| [18] | Bruni, C.; DiPillo, G.; Koch, G., Bilinear systems: an appealing class of “nearly linear” systems in theory and applications, IEEE Transactions on Automatic Control, 19, 4, 334-348 (1974) · Zbl 0285.93015 · doi:10.1109/tac.1974.1100617 |
| [19] | Patil, N. S.; Sharma, S. N., On the mathematical theory of a time-varying bilinear stochastic differential system and its application to two dynamic circuits, Transactions of the Institute of Systems, Control and Information Engineers, 27, 12, 485-492 (2014) · doi:10.5687/iscie.27.485 |
| [20] | Carravetta, F.; Germani, A.; Raimondi, N., Polynomial filtering of discrete-time stochastic linear systems with multiplicative state noise, IEEE Transactions on Automatic Control, 42, 8, 1106-1126 (1997) · Zbl 0888.93058 · doi:10.1109/9.618240 |
| [21] | Carravetta, F.; Germani, A.; Shuakayev, M. K., A new suboptimal approach to the filtering problem for bilinear stochastic differential systems, SIAM Journal on Control and Optimization, 38, 4, 1171-1203 (2000) · Zbl 0957.93079 · doi:10.1137/s0363012997320912 |
| [22] | Germani, A.; Manes, C.; Palumbo, P., Polynomial extended Kalman filter, IEEE Transactions on Automatic Control, 50, 12, 2059-2064 (2005) · Zbl 1365.93503 · doi:10.1109/tac.2005.860256 |
| [23] | Germani, A.; Manes, C.; Palumbo, P., A family of polynomial filters for discrete-time nonlinear stochastic systems, Proceedings of the 16th IFAC World Congress on Automatic Control · doi:10.3182/20050703-6-cz-1902.00215 |
| [24] | Germani, A.; Manes, C.; Palumbo, P., Filtering of stochastic nonlinear differential systems via a Carleman approximation approach, IEEE Transactions on Automatic Control, 52, 11, 2166-2172 (2007) · Zbl 1366.93653 · doi:10.1109/tac.2007.908347 |
| [25] | Cacace, F.; Cusimano, V.; Germani, A.; Palumbo, P., A state predictor for continuous-time stochastic systems, Systems and Control Letters, 98, 37-43 (2016) · Zbl 1351.93152 · doi:10.1016/j.sysconle.2016.10.004 |
| [26] | Mavelli, G.; Palumbo, P., The Carleman approximation approach to solve a stochastic nonlinear control problem, IEEE Transactions on Automatic Control, 55, 4, 976-982 (2010) · Zbl 1368.93790 · doi:10.1109/tac.2010.2041611 |
| [27] | Mavelli, G.; Palumbo, P., A Carleman approximation scheme for a stochastic optimal control problem in the continuous-time framework, Proceedings of the 17th IFAC World Congress on Automatic Control · doi:10.3182/20080706-5-kr-1001.01355 |
| [28] | Liptser, R. S.; Shiryayev, A. N., Statistics of Random Processes I and II (1977), Berlin, Germany: Springer, Berlin, Germany · Zbl 0364.60004 |
| [29] | Higham, D. J., An algorithmic introduction to numerical simulation of stochastic differential equations, SIAM Review, 43, 3, 525-546 (2001) · Zbl 0979.65007 · doi:10.1137/s0036144500378302 |
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. In some cases that data have been complemented/enhanced by data from zbMATH Open. This attempts to reflect the references listed in the original paper as accurately as possible without claiming completeness or a perfect matching.
