Second-order Hamilton-Jacobi PDE problems and certain related first-order problems. I: Approximation. (English) Zbl 1530.93547
Summary: A class of nonlinear, stochastic staticization control problems (including minimization problems with smooth, convex, coercive payoffs) driven by diffusion dynamics with constant diffusion coefficient is considered. The nonlinearities are addressed through stat duality. The second-order Hamilton-Jacobi partial differential equation (HJ PDE) is converted into a first-order HJ PDE in the dual variable, which, however, contains a correction term. Approximations to the correction term are indicated. A numerical example is included.
MSC:
| 93E20 | Optimal stochastic control |
| 49L12 | Hamilton-Jacobi equations in optimal control and differential games |
| 49L20 | Dynamic programming in optimal control and differential games |
| 93C10 | Nonlinear systems in control theory |
Keywords:
dynamic programming; Hamilton-Jacobi; partial differential equations; stochastic control; stationary action; staticizationReferences:
| [1] | Akian, M. and Fodjo, E., A probabilistic max-plus numerical method for solving stochastic control problems, in Proc. 55th IEEE CDC, , 2016, pp. 7392-7397. |
| [2] | Akian, M., Gaubert, S., and Lakhoua, A., The max-plus finite element method for solving deterministic optimal control problems: Basic properties and convergence analysis, SIAM J. Control Optim., 47 (2008), pp. 817-848, doi:10.1137/060655286. · Zbl 1157.49034 |
| [3] | Doss, H., On a probabilistic approach to the Schrödinger equation with a time-dependent potential, J. Funct. Anal., 260 (2011), pp. 1824-1835. · Zbl 1209.35160 |
| [4] | Dower, P. M. and McEneaney, W. M., Verification of stationary action trajectories via optimal control, in Proc. 2020 American Control Conference, . · Zbl 1380.49020 |
| [5] | Dower, P. M., McEneaney, W. M., and Cantoni, M., A game representation for a finite horizon state constrained continuous time linear regulator problem, Appl. Math. Optim., 88 (2023), 19, doi:10.1007/s00245-023-09972-6. · Zbl 1520.65066 |
| [6] | Dower, P. M., An adaptive max-plus eigenvector method for continuous time optimal control problems, in Numerical Methods for Optimal Control Problems, , Falcone, M., Ferretti, R., Grune, L., and McEneaney, W. M., eds., Springer, Cham, 2018, pp. 211-240. · Zbl 1416.49028 |
| [7] | Dower, P. M. and McEneaney, W. M., Solving two-point boundary value problems for a wave equation via the principle of stationary action and optimal control, SIAM J. Control Optim., 55 (2017), pp. 2151-2205, doi:10.1137/151003167. · Zbl 1380.49020 |
| [8] | Ekeland, I., Legendre duality in nonconvex optimization and calculus of variations, SIAM J. Control Optim., 15 (1977), pp. 905-934, doi:10.1137/0315058. · Zbl 0377.90089 |
| [9] | Falcone, M. and Ferretti, R., Semi-Lagrangian Approximation Schemes for Linear and Hamilton-Jacobi Equations, SIAM, Philadelphia, 2014, doi:10.1137/1.9781611973051. · Zbl 1335.65001 |
| [10] | Feynman, R. P., Space-time approach to non-relativistic quantum mechanics, Rev. Modern Phys., 20 (1948), pp. 367-387. · Zbl 1371.81126 |
| [11] | Fleming, W. H. and Soner, H. M., Controlled Markov Processes and Viscosity Solutions, 2nd ed., Springer, New York, 2006. · Zbl 1105.60005 |
| [12] | Fleming, W. H. and Rishel, R. W., Deterministic and Stochastic Optimal Control, Springer, New York, 1982. · Zbl 0323.49001 |
| [13] | Fleming, W. H., Stochastic calculus of variations and mechanics, J. Optim. Theory Appl., 41 (1983), pp. 55-74. · Zbl 0497.49013 |
| [14] | Gaubert, S., McEneaney, W. M., and Qu, Z., Curse of dimensionality reduction in max-plus based approximation methods: Theoretical estimates and improved pruning algorithms, in Proc. IEEE CDC and ECC, , 2011, pp. 1054-1061. |
| [15] | Han, S. H. and McEneaney, W. M., The principle of least action and a two-point boundary value problem in orbital mechanics, Appl. Math. Optim., 77 (2018), pp. 129-172. · Zbl 1386.49054 |
| [16] | Kaise, H. and McEneaney, W. M., Idempotent expansions for continuous-time stochastic control, SIAM J. Control Optim., 54 (2016), pp. 73-98, doi:10.1137/140971038. · Zbl 1348.49022 |
| [17] | Karatzas, I. and Shreve, S. E., Brownian Motion and Stochastic Calculus, Springer-Verlag, New York, 1987. · Zbl 0638.60065 |
| [18] | Lang, S., Fundamentals of Differential Geometry, Springer, New York, 1999. · Zbl 0932.53001 |
| [19] | McEneaney, W. M. and Zhao, R., Staticization and iterated staticization, SIAM J. Control Optim., 60 (2022), pp. 1840-1862, doi:10.1137/19M1251278. · Zbl 1496.93062 |
| [20] | McEneaney, W. M. and Dower, P. M., Staticization-based representations for Schrödinger equations driven by Coulomb potentials, in Proc. 3rd IFAC Workshop on Thermo-dynamic Foundations for a Math. Systems Theory (Louvain-la-Neuve, Belgium), , 2019, doi:10.1016/j.ifacol.2019.07.005. |
| [21] | McEneaney, W. M. and Zhao, R., Diffusion process representations for a scalar-field Schrödinger equation solution in rotating coordinates, in Numerical Methods for Optimal Control Problems, , Falcone, M., Ferretti, R., Grune, L., and McEneaney, W., eds., Springer, Cham, 2018, pp. 241-268. · Zbl 1416.49030 |
| [22] | McEneaney, W. M. and Dower, P. M., Static duality and a stationary-action application, J. Differential Equations, 264 (2018), pp. 525-549. · Zbl 1381.49036 |
| [23] | McEneaney, W. M., A stochastic control verification theorem for the dequantized Schrödinger equation not requiring a duration restriction, Appl. Math. Optim., 79 (2019), pp. 427-452. · Zbl 1420.35280 |
| [24] | McEneaney, W. M. and Dower, P. M., The principle of least action and fundamental solutions of mass-spring and \(n\)-body two-point boundary value problems, SIAM J. Control Optim., 53 (2015), pp. 2898-2933, doi:10.1137/130921908. · Zbl 1325.49055 |
| [25] | McEneaney, W. M., Max-Plus Methods for Nonlinear Control and Estimation, Birkhäuser, Boston, 2006. · Zbl 1103.93005 |
| [26] | McEneaney, W. M., Convergence rate for a curse-of-dimensionality-free method for Hamilton-Jacobi-Bellman PDEs represented as maxima of quadratic forms, SIAM J. Control Optim., 48 (2009), pp. 2651-2685, doi:10.1137/070687980. · Zbl 1203.49039 |
| [27] | Qu, Z., A max-plus based randomized algorithm for solving a class of HJB PDEs, in Proc. IEEE CDC, , 2014, pp. 1575-1580. |
| [28] | Rockafellar, R. T. and Wets, R. J., Variational Analysis, Springer-Verlag, New York, 1997. · Zbl 0888.49001 |
| [29] | Sridharan, S., McEneaney, W. M., Gu, M., and James, M. R., A reduced complexity min-plus solution method to the optimal control of closed quantum systems, Appl. Math. Optim., 70 (2014), pp. 469-510. · Zbl 1304.49058 |
| [30] | Zhang, H., Dower, P. M., and McEneaney, W. M., A pruning algorithm for managing complexity in the solution of a class of linear non-quadratic regulator problems, in Proc. Australian Control Conference, , 2013, pp. 307-312. |
| [31] | Zhao, R. and McEneaney, W. M., Iterated staticization and efficient solution of conservative and quantum systems, in Proc. SIAM Conference on Control and Its Applications, , 2019, pp. 83-90, doi:10.1137/1.9781611975758.13. · Zbl 07876445 |
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. In some cases that data have been complemented/enhanced by data from zbMATH Open. This attempts to reflect the references listed in the original paper as accurately as possible without claiming completeness or a perfect matching.
