The turnpike property and the longtime behavior of the Hamilton-Jacobi-Bellman equation for finite-dimensional LQ control problems. (English) Zbl 1505.49026
Summary: We analyze the consequences that the so-called turnpike property has on the longtime behavior of the value function corresponding to a finite-dimensional linear-quadratic optimal control problem with general terminal cost and constrained controls. We prove that, when the time horizon \(T\) tends to infinity, the value function asymptotically behaves as \(W(x) + cT + \lambda\), and we provide a control interpretation of each of these three terms, making clear the link with the turnpike property. As a by-product, we obtain the longtime behavior of the solution to the associated Hamilton-Jacobi-Bellman equation in a case where the Hamiltonian is not coercive in the momentum variable. As a result of independent interest, we showed that linear-quadratic optimal control problems with constrained control enjoy a turnpike property, also particularly when the steady optimum may saturate the control constraints.
MSC:
| 49N10 | Linear-quadratic optimal control problems |
| 49L20 | Dynamic programming in optimal control and differential games |
Keywords:
optimal control problems; longtime behavior; the turnpike property; Hamilton-Jacobi-Bellman equations; linear-quadratic (LQ)References:
| [1] | Abou-Kandil, H.; Freiling, G.; Ionescu, V.; Jank, G., Matrix Riccati equations in control and systems theory, systems and control: foundations and applications (2012), Basel: Birkhäuser, Basel · Zbl 1027.93001 |
| [2] | Anderson, BD; Moore, JB, Optimal control: linear quadratic methods (2007), North Chelmsford: Courier Corporation, North Chelmsford |
| [3] | Angeli, D.; Amrit, R.; Rawlings, JB, On average performance and stability of economic model predictive control, IEEE Trans Autom Control, 57, 1615-1626 (2011) · Zbl 1369.93209 · doi:10.1109/TAC.2011.2179349 |
| [4] | Arisawa M (1997) Ergodic problem for the Hamilton-Jacobi-Bellman equation. I. Existence of the ergodic attractor. In: Annales de l’Institut Henri Poincare (C) non linear analysis, vol 14, Elsevier, pp 415-438 · Zbl 0892.49015 |
| [5] | Arisawa M (1998) Ergodic problem for the Hamilton-Jacobi-Bellman equation. II, In: Annales de l’Institut Henri Poincare (C) Non Linear Analysis, vol 15, Elsevier, pp. 1-24 · Zbl 0903.49018 |
| [6] | Bardi, M.; Capuzzo-Dolcetta, I., Optimal control and viscosity solutions of Hamilton-Jacobi-Bellman equations (2008), Berlin: Springer, Berlin · Zbl 1134.49022 |
| [7] | Barles, G.; Ley, O.; Nguyen, T-T; Phan, TV, Large time behavior of unbounded solutions of first-order Hamilton-Jacobi equations in \({\mathbb{R}}^N\), Asymptot Anal, 112, 1-22 (2019) · Zbl 1431.35194 |
| [8] | Barles, G.; Roquejoffre, J-M, Ergodic type problems and large time behaviour of unbounded solutions of Hamilton-Jacobi equations, Commun Partial Differ. Equ., 31, 1209-1225 (2006) · Zbl 1107.35019 · doi:10.1080/03605300500361461 |
| [9] | Barles, G.; Souganidis, PE, On the large time behavior of solutions of Hamilton-Jacobi equations, SIAM J Math Anal, 31, 925-939 (2000) · Zbl 0960.70015 · doi:10.1137/S0036141099350869 |
| [10] | Bensoussan, A.; Da Prato, G.; Delfour, M.; Mitter, S., Representation and control of infinite dimensional systems, systems and control: foundations and applications (2011), Boston: Birkhäuser, Boston · Zbl 1117.93002 |
| [11] | Bensoussan, A.; Frehse, J.; Yam, SCP, The master equation in mean field theory, J Mathématiques Pures Appliquées, 103, 1441-1474 (2015) · Zbl 1325.35232 · doi:10.1016/j.matpur.2014.11.005 |
| [12] | Bensoussan, A.; Frehse, J.; Yam, SCP, On the interpretation of the master equation, Stochas Process Appl, 127, 2093-2137 (2017) · Zbl 1379.60063 · doi:10.1016/j.spa.2016.10.004 |
| [13] | Brammer, RF, Controllability in linear autonomous systems with positive controllers, SIAM J Control, 10, 339-353 (1972) · Zbl 0242.93007 · doi:10.1137/0310026 |
| [14] | Breiten, T.; Pfeiffer, L., On the turnpike property and the receding-horizon method for linear-quadratic optimal control problems, SIAM J Control Optim, 58, 1077-1102 (2020) · Zbl 1437.49047 · doi:10.1137/18M1225811 |
| [15] | Callier, FM; Winkin, J., Convergence of the time-invariant Riccati differential equation towards its strong solution for stabilizable systems, J Math Anal Appl, 192, 230-257 (1995) · Zbl 0824.93023 · doi:10.1006/jmaa.1995.1169 |
| [16] | Cannarsa, P.; Sinestrari, C., Semiconcave functions, Hamilton-Jacobi equations, and optimal control (2004), Berlin: Springer, Berlin · Zbl 1095.49003 · doi:10.1007/b138356 |
| [17] | Cardaliaguet, P.; Porretta, A., Long time behavior of the master equation in mean field game theory, Anal PDE, 12, 1397-1453 (2019) · Zbl 1428.35607 · doi:10.2140/apde.2019.12.1397 |
| [18] | Crandall, MG; Ishii, H.; Lions, P-L, User’s guide to viscosity solutions of second order partial differential equations, Bull Am Math Soc, 27, 1-67 (1992) · Zbl 0755.35015 · doi:10.1090/S0273-0979-1992-00266-5 |
| [19] | Crandall, MG; Lions, P-L, Viscosity solutions of Hamilton-Jacobi equations, Trans Am Math Soc, 277, 1-42 (1983) · Zbl 0599.35024 · doi:10.1090/S0002-9947-1983-0690039-8 |
| [20] | Esteve C, Geshkovski B, Pighin D, Zuazua E (2020) Turnpike in lipschitz-nonlinear optimal control. arXiv:2011.11091 · Zbl 1484.49006 |
| [21] | Evans, LC, Partial differential equations (2010), Providence: American Mathematical Soc, Providence · Zbl 1194.35001 |
| [22] | Faulwasser, T.; Kellett, CM, On continuous-time infinite horizon optimal control-dissipativity, stability, and transversality, Automatica, 134, 109907 (2021) · Zbl 1478.49005 · doi:10.1016/j.automatica.2021.109907 |
| [23] | Fujita, Y.; Ishii, H.; Loreti, P., Asymptotic solutions of Hamilton-Jacobi equations in Euclidean n space, Indiana Univ Math J, 2006, 1671-1700 (2006) · Zbl 1112.35041 · doi:10.1512/iumj.2006.55.2813 |
| [24] | Grüne, L., Approximation properties of receding horizon optimal control, Jahresber Deutsch Math Verein, 118, 3-37 (2016) · Zbl 1336.93060 · doi:10.1365/s13291-016-0134-5 |
| [25] | Grüne L (2021) Dissipativity and optimal control. arXiv:2101.12606 |
| [26] | Grüne, L.; Guglielmi, R., Turnpike properties and strict dissipativity for discrete time linear quadratic optimal control problems, SIAM J Control Optim, 56, 1282-1302 (2018) · Zbl 1387.49027 · doi:10.1137/17M112350X |
| [27] | Grüne, L.; Guglielmi, R., On the relation between turnpike properties and dissipativity for continuous time linear quadratic optimal control problems, Math Control Rel Fields, 11, 169 (2021) · Zbl 1462.49058 · doi:10.3934/mcrf.2020032 |
| [28] | Grüne, L.; Müller, MA, On the relation between strict dissipativity and turnpike properties, Syst Control Lett, 90, 45-53 (2016) · Zbl 1335.93117 · doi:10.1016/j.sysconle.2016.01.003 |
| [29] | Grüne L, Pannek J (2017) Nonlinear model predictive control. In: Nonlinear Model Predictive Control, Springer, pp 45-69 · Zbl 1429.93003 |
| [30] | Ishii, H., Asymptotic solutions for large time of Hamilton-Jacobi equations, Int Congr Math, 3, 213-227 (2006) · Zbl 1098.35029 |
| [31] | Ishii, H., Asymptotic solutions for large time of Hamilton-Jacobi equations in Euclidean \(n\) space, Ann l’IHP Analyse Non linéaire, 25, 231-266 (2008) · Zbl 1145.35035 · doi:10.1016/j.anihpc.2006.09.002 |
| [32] | Ishii H (2013) A short introduction to viscosity solutions and the large time behavior of solutions of Hamilton-Jacobi equations, In: Hamilton-Jacobi Equations: Approximations, Numerical Analysis and Applications, Springer, pp 111-249 · Zbl 1269.49044 |
| [33] | Kouhkouh H (2018) Dynamic programming interpretation of turnpike and Hamilton-Jacobi-Bellman equation. Master thesis, Paris-Saclay University. http://bit.ly/2R7soRx |
| [34] | Kwakernaak, H.; Sivan, R., Linear optimal control systems (1972), New York: Wiley, New York · Zbl 0276.93001 |
| [35] | Lee, EB; Markus, L., Foundations of optimal control theory (1967), Florida: Robert E. Krieger Publishing Company, Florida · Zbl 0159.13201 |
| [36] | Lions, P-L, Generalized solutions of Hamilton-Jacobi equations (1982), London: Pitman, London · Zbl 0497.35001 |
| [37] | Pighin D (2020) Nonuniqueness of minimizers for semilinear optimal control problems. arXiv:2002.04485 · Zbl 1521.49004 |
| [38] | Pighin D (2020) The turnpike property in semilinear control. arXiv:2004.03269 · Zbl 1473.49044 |
| [39] | Pighin D, Sakamoto N (2020) The turnpike with lack of observability. arXiv:2007.14081 |
| [40] | Porretta, A.; Zuazua, E., Long time versus steady state optimal control, SIAM J Control Optim, 51, 4242-4273 (2013) · Zbl 1287.49006 · doi:10.1137/130907239 |
| [41] | Porretta A, Zuazua E (2016) Remarks on long time versus steady state optimal control. In: Mathematical paradigms of climate science, Springer, pp 67-89 · Zbl 1369.49008 |
| [42] | Roquejoffre, J-M, Convergence to steady states or periodic solutions in a class of Hamilton-Jacobi equations, J Mathématiques pures et appliquées, 80, 85-104 (2001) · Zbl 0979.35033 · doi:10.1016/S0021-7824(00)01183-1 |
| [43] | Sakamoto, N.; van der Schaft, AJ, Analytical approximation methods for the stabilizing solution of the Hamilton-Jacobi equation, IEEE Trans Autom Control, 53, 2335-2350 (2008) · Zbl 1367.93535 · doi:10.1109/TAC.2008.2006113 |
| [44] | Trélat, E., Contrôle optimal: théorie & applications (2005), Paris: Vuibert, Paris · Zbl 1112.49001 |
| [45] | Trélat, E.; Zhang, C., Integral and measure-turnpike properties for infinite-dimensional optimal control systems, Math Control Signals Syst, 30, 3 (2018) · Zbl 1394.49018 · doi:10.1007/s00498-018-0209-1 |
| [46] | Trélat, E.; Zhang, C.; Zuazua, E., Steady-state and periodic exponential turnpike property for optimal control problems in Hilbert spaces, SIAM J Control Optim, 56, 1222-1252 (2018) · Zbl 1387.49042 · doi:10.1137/16M1097638 |
| [47] | Trélat, E.; Zuazua, E., The turnpike property in finite-dimensional nonlinear optimal control, J Differ Equ, 258, 81-114 (2015) · Zbl 1301.49010 · doi:10.1016/j.jde.2014.09.005 |
| [48] | Willems, J., Least squares stationary optimal control and the algebraic Riccati equation, IEEE Trans Autom Control, 16, 621-634 (1971) · doi:10.1109/TAC.1971.1099831 |
| [49] | Zanon, M.; Faulwasser, T., Economic MPC without terminal constraints: gradient-correcting end penalties enforce asymptotic stability, J Process Control, 63, 1-14 (2018) · doi:10.1016/j.jprocont.2017.12.005 |
| [50] | Zanon, M.; Gros, S.; Diehl, M., A tracking MPC formulation that is locally equivalent to economic MPC, J Process Control, 45, 30-42 (2016) · doi:10.1016/j.jprocont.2016.06.006 |
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. In some cases that data have been complemented/enhanced by data from zbMATH Open. This attempts to reflect the references listed in the original paper as accurately as possible without claiming completeness or a perfect matching.
