Optimal control of stochastic delay differential equations and applications to path-dependent financial and economic models. (English) Zbl 07854558
A class of optimal control problems of stochastic differential delay equations is considered. First, the problem is rewritten in a suitable infinite-dimensional Hilbert space. Then, using the dynamic programming approach, the value function of the problem is characterized as the unique viscosity solution of the associated infinite-dimensional Hamilton-Jacobi-Bellman equation. Finally, the authors prove a \(C^{1, \alpha}\)-partial regularity of the value function. The optimal control problem at hand being not Markovian due to the delay, in order to regain Markovianity and approach the problem by dynamic programming, following a well-known procedure ([A. Bensoussan et al., Representation and control of infinite dimensional systems. 2nd ed. Boston, MA: Birkhäuser (2007; Zbl 1117.93002)] for deterministic delay equations and [A. Chojnowska Michalik, Bull. Acad. Pol. Sci., Sér. Sci. Math. Astron. Phys. 26, 635–642 (1978; Zbl 0415.60057); G. Da Prato and J. Zabczyk, Stochastic equations in infinite dimensions. 2nd ed. Cambridge: Cambridge University Press (2014; Zbl 1317.60077); S. Federico et al., SIAM J. Control Optim. 48, No. 8, 4910–4937 (2010; Zbl 1208.49048)] for the stochastic case, the state equation is reformulated by lifting it to an infinite-dimensional space. The state equation is rewritten using an adequate maximal dissipative operator, the authors introducing an adequate associated operator which is shown to satisfy the weak \(B\)-condition.This is needed to be in the framework of the theory of viscosity solutions to the asscoiated HJB equation. Then estimates for solutions of the state equation, the cost functional and the value function are proven, as well as regularity properties. The value function \(V\) is characterized as the unique \(B\)-continuous viscosity solution to the associated HJB equation, thus providing existence and uniqueness results for fully nonlinear HJB equations in Hilbert spaces related to a general class of stochastic optimal control problems with delays involving controls in the diffusion coefficient. Note that besides the innovative results and so clear a writing, a very nice reliable synthetic introduction to important questions and paths to such issues is here provided. The results are applied to path dependent financial and economic problems, two examples are provided (one is a Merton-like problem with path dependent coefficients, and the other an optimal advertising with delays as in [F. Gozzi and C. Marinelli, Lect. Notes Pure Appl. Math. 245, 133–148 (2006; Zbl 1107.93035)]) and a section is dedicated to possible applications.
Reviewer: Lisa Morhaim (Paris)
MSC:
| 49K45 | Optimality conditions for problems involving randomness |
| 49K27 | Optimality conditions for problems in abstract spaces |
| 90C39 | Dynamic programming |
| 35F21 | Hamilton-Jacobi equations |
| 35D40 | Viscosity solutions to PDEs |
| 49S05 | Variational principles of physics |
| 93E20 | Optimal stochastic control |
| 60H15 | Stochastic partial differential equations (aspects of stochastic analysis) |
| 49L20 | Dynamic programming in optimal control and differential games |
| 35R15 | PDEs on infinite-dimensional (e.g., function) spaces (= PDEs in infinitely many variables) |
| 49L12 | Hamilton-Jacobi equations in optimal control and differential games |
Keywords:
stochastic optimal control; viscosity solutions in Hilbert spaces; partial regularity; stochastic delay equations; path-dependent equations; Merton problemReferences:
| [1] | Addona, D., Bandini, E., and Masiero, F., A nonlinear Bismut-Elworthy formula for HJB equations with quadratic Hamiltonian in Banach spaces, Nonlinear Differ. Equ. Appl., 27 (2020), 37. · Zbl 1443.60058 |
| [2] | Bambi, M., Fabbri, G., and Gozzi, F., Optimal policy and consumption smoothing effects in the time-to-build AK model, Econom. Theory, 50 (2012), pp. 635-669. · Zbl 1246.91082 |
| [3] | Bambi, M., Di Girolami, C., Federico, S., and Gozzi, F., Generically distributed investments on flexible projects and endogenous growth, Econom. Theory, 63 (2017), pp. 521-558. · Zbl 1405.91398 |
| [4] | Barbu, V. and Da Prato, G., Hamilton-Jacobi Equations in Hilbert Spaces, , Pitman, Boston, 1983. · Zbl 0508.34001 |
| [5] | Bardi, M. and Capuzzo-Dolcetta, I., Optimal Control and Viscosity Solutions of Hamilton-Jacobi-Bellman Equations, , Birkhäuser, Boston, 1997. · Zbl 0890.49011 |
| [6] | Bayraktar, E. and Keller, C., Path-dependent Hamilton-Jacobi equations in infinite dimensions, J. Funct. Anal., 275 (2018), pp. 2096-2161. · Zbl 1401.35328 |
| [7] | Bayraktar, E. and Keller, C., Path-dependent Hamilton-Jacobi equations with super-quadratic growth in the gradient and the vanishing viscosity method, SIAM J. Control Optim., 60 (2022), pp. 1690-1711, doi:10.1137/21M1395557. · Zbl 1492.60183 |
| [8] | Bensoussan, A., Da Prato, G., Delfour, M. C., and Mitter, S. K., Representation and Control of Infinite Dimensional Systems, 2nd ed., , Birkhäuser, Boston, 2007. · Zbl 1117.93002 |
| [9] | Bensoussan, A. and Yam, P., Control problem on space of random variables and master equation, ESAIM Control Optim. Calc. Var., 25 (2019), 10. · Zbl 1450.35305 |
| [10] | Bensoussan, A., Graber, P., and Yam, P., Control on Hilbert Spaces and Application to Mean Field Type Control Theory, preprint, https://arxiv.org/abs/2005.10770, 2020. |
| [11] | Biffis, E., Gozzi, F., and Prosdocimi, C., Optimal portfolio choice with path dependent labor income: The infinite horizon case, SIAM J. Control Optim., 58 (2020), pp. 1906-1938, doi:10.1137/19M1259687. · Zbl 1451.49031 |
| [12] | Biagini, S., Gozzi, F., and Zanella, M., Robust portfolio choice with sticky wages, SIAM J. Financial Math., 13 (2022), pp. 1004-1039, doi:10.1137/21M1429722. · Zbl 1498.91377 |
| [13] | Djehiche, B., Gozzi, F., Zanco, G., and Zanella, M., Optimal portfolio choice with path dependent benchmarked labor income: A mean field model, Stochastic Process. Appl., 145 (2022), pp. 48-85. · Zbl 1489.34117 |
| [14] | Caffarelli, L. and Cabré, X., Fully Nonlinear Elliptic Equations, , American Mathematical Society, Providence, RI, 1995. · Zbl 0834.35002 |
| [15] | Caffarelli, L., Crandall, M. G., Kocan, M., and Święch, A., On viscosity solutions of fully nonlinear equations with measurable ingredients, Comm. Pure Appl. Math., 49 (1996), pp. 365-397. · Zbl 0854.35032 |
| [16] | Cannarsa, P. and Frankowska, H., Value function and optimality conditions for semilinear control problems, Appl. Math. Optim., 26 (1992), pp. 139-169. · Zbl 0765.49001 |
| [17] | Carlier, G. and Tahraoui, R., Hamilton-Jacobi-Bellman equations for the optimal control of a state equation with memory, ESAIM Control Optim. Calc. Var., 16 (2010), pp. 744-763. · Zbl 1195.49032 |
| [18] | Cont, R. and Fournié, D., Change of variable formulas for non-anticipative functionals on path space, J. Funct. Anal., 259 (2010), pp. 1043-1072. · Zbl 1201.60051 |
| [19] | Cont, R. and Fournié, D., Functional Itô calculus and stochastic integral representation of martingales, Ann. Probab., 41 (2013), pp. 109-133. · Zbl 1272.60031 |
| [20] | Chojnowska-Michalik, A., Representation theorem for general stochastic delay equations, Bull. Acad. Polon. Sci. Sér. Sci. Math. Astronom. Phys., 26 (1978), pp. 635-642. · Zbl 0415.60057 |
| [21] | Cosso, A., Federico, S., Gozzi, F., Rosestolato, M., and Touzi, N., Path-dependent equations and viscosity solutions in infinite dimension, Ann. Probab., 46 (2018), pp. 126-174. · Zbl 1516.35192 |
| [22] | Crandall, M. G. and Lions, P. L., Viscosity solutions of Hamilton-Jacobi equations in infinite dimensions. IV. Hamiltonians with unbounded linear terms, J. Funct. Anal., 90 (1990), pp. 237-283. · Zbl 0739.49016 |
| [23] | Crandall, M. G. and Lions, P. L., Viscosity solutions of Hamilton-Jacobi equations in infinite dimensions. V. Unbounded linear terms and \(B\)-continuous solutions, J. Funct. Anal., 97 (1991), pp. 417-465. · Zbl 0739.49017 |
| [24] | Da Prato, G. and Zabczyk, J., Ergodicity for Infinite-Dimensional Systems, , Cambridge University Press, Cambridge, 1996. · Zbl 0849.60052 |
| [25] | Da Prato, G. and Zabczyk, J., Second Order Partial Differential Equations in Hilbert Spaces, , Cambridge University Press, Cambridge, 2002. · Zbl 1012.35001 |
| [26] | Da Prato, G. and Zabczyk, J., Stochastic Equations in Infinite Dimensions, , Cambridge University Press, Cambridge, 2014. · Zbl 1317.60077 |
| [27] | de Feo, F. and Święch, A., Optimal Control of Stochastic Delay Differential Equations: Optimal Feedback Controls, preprint, https://arxiv.org/abs/2309.05029, 2023. |
| [28] | Di Giacinto, M., Federico, S., and Gozzi, F., Pension funds with a minimum guarantee: A stochastic control approach, Finance Stoch., 15 (2011), pp. 297-342. · Zbl 1303.91155 |
| [29] | Elsanosi, I., Øksendal, B., and Sulem, A., Some solvable stochastic control problems with delay, Stochastics Rep., 71 (2000), pp. 69-89. · Zbl 0999.93072 |
| [30] | Engel, K. J. and Nagel, R., One-Parameter Semigroups for Linear Evolution Equations, , Springer, Berlin, 2000. · Zbl 0952.47036 |
| [31] | Ekren, I., Keller, C., Touzi, N., and Zhang, J., On viscosity solutions of path dependent PDEs, Ann. Probab., 42 (2014), pp. 204-236. · Zbl 1320.35154 |
| [32] | Ekren, I., Touzi, N., and Zhang, J., Viscosity solutions of fully nonlinear parabolic path dependent PDEs: Part I, Ann. Probab., 44 (2016), pp. 1212-1253. · Zbl 1375.35250 |
| [33] | Ekren, I., Touzi, N., and Zhang, J., Viscosity solutions of fully nonlinear parabolic path dependent PDEs: Part II, Ann. Probab., 44 (2016), pp. 2507-2553. · Zbl 1394.35228 |
| [34] | Fabbri, G. and Gozzi, F., Solving optimal growth models with vintage capital: The dynamic programming approach, J. Econom. Theory, 143 (2008), pp. 331-373. · Zbl 1151.91069 |
| [35] | Fabbri, G., Gozzi, F., and Święch, A., Verification theorem and construction of \(\varepsilon \)-optimal controls for control of abstract evolution equations, J. Convex Anal., 17 (2010), pp. 611-642. · Zbl 1193.35246 |
| [36] | Fabbri, G., Gozzi, F., and Święch, A., Stochastic Optimal Control in Infinite Dimension. Dynamic Programming and HJB Equations. With a Contribution by Marco Fuhrman and Gianmario Tessitore, , Springer, Cham, 2017. · Zbl 1379.93001 |
| [37] | Faggian, S., Regular solutions of first-order Hamilton-Jacobi equations for boundary control problems and applications to economics, Appl. Math. Optim., 51 (2005), pp. 123-162. · Zbl 1078.49026 |
| [38] | Faggian, S., Hamilton-Jacobi equations arising from boundary control problems with state constraints, SIAM J. Control Optim., 47 (2008), pp. 2157-2178, doi:10.1137/070683738. · Zbl 1167.49005 |
| [39] | Faggian, S. and Gozzi, F., Optimal investment models with vintage capital: Dynamic programming approach, J. Math. Econom., 46 (2010), pp. 416-437. · Zbl 1196.49022 |
| [40] | Federico, S., A stochastic control problem with delay arising in a pension fund model, Finance Stoch., 15 (2011), pp. 421-459. · Zbl 1302.93238 |
| [41] | Federico, S., Gassiat, P., and Gozzi, F., Utility maximization with current utility on the wealth: Regularity of solutions to the HJB equation, Finance Stoch., 19 (2015), pp. 415-448. · Zbl 1307.93453 |
| [42] | Federico, S., Goldys, B., and Gozzi, F., HJB equations for the optimal control of differential equations with delays and state constraints, I: Regularity of viscosity solutions, SIAM J. Control Optim., 48 (2010), pp. 4910-4937, doi:10.1137/09076742X. · Zbl 1208.49048 |
| [43] | Federico, S., Goldys, B., and Gozzi, F., HJB equations for the optimal control of differential equations with delays and state constraints, II: Verification and optimal feedbacks, SIAM J. Control Optim., 49 (2011), pp. 2378-2414, doi:10.1137/100804292. · Zbl 1242.49058 |
| [44] | Federico, S. and Gozzi, F., Verification theorems for stochastic optimal control problems in Hilbert spaces by means of a generalized Dynkin formula, Ann. Appl. Probab., 28 (2018), pp. 3558-3599. · Zbl 1405.93231 |
| [45] | Federico, S. and Tacconi, E., Dynamic programming for optimal control problems with delays in the control variable, SIAM J. Control Optim., 52 (2014), pp. 1203-1236, doi:10.1137/110840649. · Zbl 1293.49060 |
| [46] | Federico, S. and Tankov, P., Finite-dimensional representations for controlled diffusions with delay, Appl. Math. Optim., 71 (2015), pp. 165-194. · Zbl 1310.60078 |
| [47] | Flandoli, F. and Zanco, G., An infinite-dimensional approach to path-dependent Kolmogorov’s equations, Ann. Probab., 44 (2016), pp. 2643-2693. · Zbl 1356.60101 |
| [48] | Fuhrman, M., Masiero, F., and Tessitore, G., Stochastic equations with delay: Optimal control via BSDEs and regular solutions of Hamilton-Jacobi-Bellman equations, SIAM J. Control Optim., 48 (2010), pp. 4624-4651, doi:10.1137/080730354. · Zbl 1213.60109 |
| [49] | Fuhrman, M. and Tessitore, G., Nonlinear Kolmogorov equations in infinite dimensional spaces: The backward stochastic differential equations approach and applications to optimal control, Ann. Probab., 30 (2002), pp. 1397-1465. · Zbl 1017.60076 |
| [50] | Fuhrman, M. and Tessitore, G., Infinite horizon backward stochastic differential equations and elliptic equations in Hilbert spaces, Ann. Probab., 32 (2004), pp. 607-660. · Zbl 1046.60061 |
| [51] | Gangbo, W. and Mészáros, A., Global well-posedness of master equations for deterministic displacement convex potential mean field games, Comm. Pure Appl. Math., 75 (2022), pp. 2685-2801. · Zbl 1510.35342 |
| [52] | Goldys, B. and Gozzi, F., Second order parabolic Hamilton-Jacobi-Bellman equations in Hilbert spaces and stochastic control: \(L_\mu^2\) approach, Stochastic Process. Appl., 116 (2006), pp. 1932-1963. · Zbl 1111.49021 |
| [53] | Gozzi, F. and Marinelli, C., Stochastic optimal control of delay equations arising in advertising models, in Stochastic Partial Differential Equations and Applications-VII, , Chapman & Hall/CRC, Boca Raton, FL, 2006, pp. 133-148. · Zbl 1107.93035 |
| [54] | Gozzi, F., Marinelli, C., and Savin, S., On controlled linear diffusions with delay in a model of optimal advertising under uncertainty with memory effects, J. Optim. Theory Appl., 142 (2009), pp. 291-321. · Zbl 1175.90245 |
| [55] | Gozzi, F. and Masiero, F., Stochastic optimal control with delay in the control I: Solving the HJB equation through partial smoothing, SIAM J. Control Optim., 55 (2017), pp. 2981-3012, doi:10.1137/16M1070128. · Zbl 1375.93140 |
| [56] | Gozzi, F. and Masiero, F., Stochastic optimal control with delay in the control II: Verification theorem and optimal feedbacks, SIAM J. Control Optim., 55 (2017), pp. 3013-3038, doi:10.1137/16M1073637. · Zbl 1371.93216 |
| [57] | Gozzi, F. and Masiero, F., Errata: Stochastic optimal control with delay in the control I: Solving the HJB equation through partial smoothing, and stochastic optimal control with delay in the control II: Verification theorem and optimal feedbacks, SIAM J. Control Optim., 59 (2021), pp. 3096-3101, doi:10.1137/21M1407434. · Zbl 1476.93159 |
| [58] | Gozzi, F., Święch, A., and Zhou, X. Y., A corrected proof of the stochastic verification theorem within the framework of viscosity solutions, SIAM J. Control Optim., 43 (2005), pp. 2009-2019, doi:10.1137/S0363012903428184. · Zbl 1116.93053 |
| [59] | Gozzi, F., Święch, A., and Zhou, X. Y., Erratum: “A corrected proof of the stochastic verification theorem within the framework of viscosity solutions”, SIAM J. Control Optim., 48 (2010), pp. 4177-4179, doi:10.1137/090775567. · Zbl 1202.93171 |
| [60] | Guyon, J. and Lekeufack, J., Volatility is (mostly) path-dependent, Quant. Finance, 23 (2023), pp. 1221-1258, doi:10.1080/14697688.2023.2221281. · Zbl 1522.91275 |
| [61] | Larssen, B., Dynamic programming in stochastic control of systems with delay, Stoch. Stoch. Rep., 74 (2002), pp. 651-673. · Zbl 1022.34075 |
| [62] | Larssen, B. and Risebro, N. H., When are HJB-equations in stochastic control of delay systems finite dimensional?, Stochastic Anal. Appl., 21 (2003), pp. 643-671. · Zbl 1052.60051 |
| [63] | Li, X. J. and Yong, J. M., Optimal Control Theory for Infinite-Dimensional Systems, , Birkhäuser, Boston, 1995. |
| [64] | Lions, P. L., Viscosity solutions of fully nonlinear second-order equations and optimal stochastic control in infinite dimensions. I. The case of bounded stochastic evolutions, Acta Math., 161 (1988), pp. 243-278. · Zbl 0757.93082 |
| [65] | Masiero, F., Stochastic optimal control problems and parabolic equations in Banach spaces, SIAM J. Control Optim., 47 (2008), pp. 251-300, doi:10.1137/050632725. · Zbl 1157.93043 |
| [66] | Masiero, F. and Tessitore, G., Partial smoothing of delay transition semigroups acting on special functions, J. Differential Equations, 316 (2022), pp. 599-640. · Zbl 1491.60087 |
| [67] | Mayorga, S. and Święch, A., Finite dimensional approximations of Hamilton-Jacobi-Bellman equations for stochastic particle systems with common noise, SIAM J. Control Optim., 61 (2023), pp. 820-851, doi:10.1137/22M1489186. · Zbl 1512.35155 |
| [68] | Merton, R., Lifetime portfolio selection under uncertainty: The continuous-time case, Rev. Econom. Statist., 51 (1969), pp. 247-257. |
| [69] | Mohammed, S. E. A., Stochastic Functional Differential Equations, , Pitman, Boston, 1984. · Zbl 0584.60066 |
| [70] | Mohammed, S. E. A., Stochastic differential systems with memory: Theory, examples and applications, in Stochastic Analysis and Related Topics VI, , Decreusefond, L., Gjerde, J., Øksendal, B., and Üstünel, A. S., eds., Birkhäuser, Boston, 1998, pp. 1-77. · Zbl 0901.60030 |
| [71] | Nerlove, M. and Arrow, J. K., Optimal advertising policy under dynamic conditions, Economica, 29 (1962), pp. 129-142. |
| [72] | Pang Tao, T. and Yong, Y., A new stochastic model for stock price with delay effects, in Proceedings of the Conference on Control and its Applications, , Society for Industrial and Applied Mathematics, Philadelphia, 2019, pp. 110-117, doi:10.1137/1.9781611975758.17. · Zbl 07876449 |
| [73] | Pardoux, É. and Peng, S. G., Adapted solution of a backward stochastic differential equation, Systems Control Lett., 14 (1990), pp. 55-61. · Zbl 0692.93064 |
| [74] | Pham, H., Continuous-Time Stochastic Control and Optimization with Financial Applications, , Springer, Berlin, 2009. · Zbl 1165.93039 |
| [75] | Protter, P. E., Stochastic Integration and Differential Equations, 2nd ed., Springer-Verlag, Berlin, Heidelberg, New York, 2003. |
| [76] | Ren, Z., Touzi, N., and Zhang, J., Comparison of viscosity solutions of fully nonlinear degenerate parabolic path-dependent PDEs, SIAM J. Math. Anal., 49 (2017), pp. 4093-4116, doi:10.1137/16M1090338. · Zbl 1516.35194 |
| [77] | Ren, Z. and Rosestolato, M., Viscosity solutions of path-dependent PDEs with randomized time, SIAM J. Math. Anal., 52 (2020), pp. 1943-1979, doi:10.1137/18M122666X. · Zbl 1445.35207 |
| [78] | Revuz, D. and Yor, M., Continuous Martingales and Brownian Motion, 3rd ed., , Springer, Berlin, 1999. · Zbl 0917.60006 |
| [79] | Rosestolato, M. and Święch, A., Partial regularity of viscosity solutions for a class of Kolmogorov equations arising from mathematical finance, J. Differential Equations, 262 (2017), pp. 1897-1930. · Zbl 1362.35327 |
| [80] | Stannat, W. and Wessels, L., Necessary and Sufficient Conditions for Optimal Control of Semilinear Stochastic Partial Differential Equations, preprint, https://arxiv.org/abs/2112.09639, 2021. |
| [81] | Święch, A., \(W^{1,p}\)-interior estimates for solutions of fully nonlinear, uniformly elliptic equations, Adv. Differential Equations, 2 (1997), pp. 1005-1027. · Zbl 1023.35509 |
| [82] | Święch, A., Pointwise properties of \(L^p\)-viscosity solutions of uniformly elliptic equations with quadratically growing gradient terms, Discrete Contin. Dyn. Syst., 40 (2020), pp. 2945-2962. · Zbl 1435.35119 |
| [83] | Vinter, R. B. and Kwong, R. H., The infinite time quadratic control problem for linear systems with state and control delays: An evolution equation approach, SIAM J. Control Optim., 19 (1981), pp. 139-153, doi:10.1137/0319011. · Zbl 0465.93043 |
| [84] | Yong, J. and Zhou, X. Y., Stochastic Controls, Hamiltonian Systems and HJB Equations, , Springer, New York, 1999. · Zbl 0943.93002 |
| [85] | Zhou, J., A class of infinite-horizon stochastic delay optimal control problems and a viscosity solution to the associated HJB equation, ESAIM Control Optim. Calc. Var., 24 (2018), pp. 639-676. · Zbl 1401.93234 |
| [86] | Zhou, J., Delay optimal control and viscosity solutions to associated Hamilton-Jacobi-Bellman equations, Internat. J. Control, 92 (2019), pp. 2263-2273. · Zbl 1423.49027 |
| [87] | Zhou, J., A notion of viscosity solutions to second-order Hamilton-Jacobi-Bellman equations with delays, Internat. J. Control, 95 (2022), pp. 2611-2631. · Zbl 1501.93165 |
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. In some cases that data have been complemented/enhanced by data from zbMATH Open. This attempts to reflect the references listed in the original paper as accurately as possible without claiming completeness or a perfect matching.
