×

Multitime dynamic programming for multiple integral actions. (English) Zbl 1234.49027

Summary: This paper introduces a new type of dynamic programming PDE for optimal control problems with performance criteria involving multiple integrals. The main novel feature of the multitime dynamic programming PDE, relative to the standard Hamilton-Jacobi-Bellman PDE, is that it is connected to the multitime maximum principle and is of divergence type. Introducing a generating vector field for the maximum value function, we present an interesting and useful connection between the multitime maximum principle and the multitime dynamic programming, characterizing the optimal control by means of a multitime Hamilton-Jacobi-Bellman (divergence) PDE that may be viewed as a feedback law. Section 1 recalls the multitime maximum principle. Section 2 shows how a multitime control dynamics determines the multitime Hamilton-Jacobi-Bellman PDE via a generating vector field of the value function. Section 3 gives an example of two-time dynamics with nine velocities proving that our theory works well. Section 4 shows that the Hamilton PDEs are characteristic PDEs of multitime Hamilton-Jacobi PDE and that the costates in the multitime maximum principle are in fact gradients of the components of the generating vector field.

MSC:

49L20 Dynamic programming in optimal control and differential games
49K20 Optimality conditions for problems involving partial differential equations
35Q93 PDEs in connection with control and optimization
Full Text: DOI

References:

[1] Bardi M., Capuzzo-Dolcetta I.: Optimal Control and Viscosity Solutions of Hamilton-Jacobi-Bellman Equations. Birkhauser, Basel, Boston (1997) · Zbl 0890.49011
[2] Barles G., Tourin A.: Commutation properties of semigroups for first order Hamilton-Jacobi equations and application to multi-time equations. Indiana Univ. Math. J. 50(4), 1523–1544 (2001) · Zbl 1044.49022 · doi:10.1512/iumj.2001.50.1925
[3] Barron N., Jensen R.: The Pontryaguin maximum principle from dynamic programming and viscosity solutions to first-order partial differential equations. Trans. AMS 298, 635–641 (1986) · Zbl 0618.49011 · doi:10.1090/S0002-9947-1986-0860384-4
[4] Belbas S.A.: Multi-time dynamic programming and Riccati equations. Int. J. Control 71(1), 61–78 (1998) · Zbl 0942.49025 · doi:10.1080/002071798221920
[5] Pardalos, P., Grundel, D., Murphey, R., Prokopyev, O. (eds): Cooperative Networks, Control and Optimization. Edward Elgar Publishing, Ann Arbor, MI (2008) · Zbl 1110.93005
[6] Pardalos P., Yatsenko V.: Optimization and Control of Bilinear Systems. Springer, Berlin (2009) · Zbl 1181.93001
[7] Udrişte, C.: Multi-time maximum principle, short communication. In: International Congress of Mathematicians, Madrid, Aug 22–30, ICM Abstracts, pp. 47 (2006)
[8] Udrişte, C.: Controllability and observability of multitime linear PDE systems. In: Proceedings of The Sixth Congress of Romanian Mathematicians, vol. 1, pp. 313–319. Bucharest, Romania (2007)
[9] Udrişte, C.: Multi-time stochastic control theory, selected topics on circuits, systems, electronics, control & signal processing. In: Proceedings of the 6-th WSEAS International Conference on Circuits, Systems, Electronics, Control & Signal Processing (CSECS’07), pp. 171–176. Cairo, Egypt, Dec 29–31 (2007)
[10] Udrişte C.: Multitime controllability, observability and bang-bang principle. J. Optim. Theory Appl. 139(1), 141–157 (2008). doi: 10.1007/s10957-008-9430-2 · Zbl 1156.93013 · doi:10.1007/s10957-008-9430-2
[11] Udrişte C.: Simplified multitime maximum principle. Balk. J. Geom. Appl. 14(1), 102–119 (2009) · Zbl 1180.49023
[12] Udrişte C.: Nonholonomic approach of multitime maximum principle. Balk. J. Geom. Appl. 14(2), 111–126 (2009)
[13] Udrişte C., Matei L.: Lagrange-Hamilton Theories (in Romanian), Monographs and Textbooks 8. Geometry Balkan Press, Bucharest (2008)
[14] Udrişte C., Ţevy I.: Multi-Time Euler-Lagrange-Hamilton theory. WSEAS Trans. Math. 6(6), 701–709 (2007)
[15] Udrişte C., Ţevy I.: Multitime linear-quadratic regulator problem based on curvilinear integral. Balk. J. Geom. Appl. 14(2), 127–137 (2009)
[16] Udrişte, C., Ţevy, I.: Multitime dynamic programming for curvilinear integral actions. J. Optim. Theory Appl. 146, Online First (2010)
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. In some cases that data have been complemented/enhanced by data from zbMATH Open. This attempts to reflect the references listed in the original paper as accurately as possible without claiming completeness or a perfect matching.