Abstract
The necessary conditions for an optimal control of a stochastic control problem with recursive utilities are investigated. The first-order condition is the well-known Pontryagin-type maximum principle. When such a first-order necessary condition is singular in some sense, certain type of the second-order necessary condition will come in naturally. The aim of this paper is to explore such kind of conditions for our optimal control problem.
Similar content being viewed by others
References
Chen, Z., Epstein, L.: Ambiguity, risk, and asset returns in continuous time. Econometrica 70(4), 1403–1443 (2002)
Kushner, H.: Necessary conditions for continuous parameter stochastic optimization problems. SIAM J. Control Optim. 10(3), 550–565 (1972)
Bensoussan, A.: Lectures on Stochastic Control in Nonlinear Filtering and Stochastic Control. Lectures Notes in Mathematics. Springer, Berlin (1982)
Bismut, J.M.: An introductory approach to duality in optimal stochastic control. SIAM Rev. 20(1), 62–78 (1978)
Peng, S.: A general stochastic maximum principle for optimal control problems. SIAM J. Control Optim. 28(4), 966–979 (1990)
Peng, S.: Backward stochastic differential equations and applications to optimal control. Appl. Math. Optim. 27(2), 125–144 (1993)
Dokuchaev, N., Zhou, X.Y.: Stochastic controls with terminal contingent conditions. J. Math. Anal. Appl. 238(1), 143–165 (1999)
Shi, J., Wu, Z.: The maximum principle for fully coupled forward–backward stochastic control system. Acta Autom. Sin. 32(2), 161 (2006)
Wu, Z.: Maximum principle for optimal control problem of fully coupled forward–backward stochastic systems. Syst. Sci. Math. Sci. 3, 249–259 (1998)
Peng, S.: Open problems on backward stochastic differential equations. In: Chen, S., Li, X., Yong, J., Zhou, X.Y. (eds.) Control of Distributed Parameter and Stochastic Systems, pp. 265–273. Springer, New York (1999)
Wu, Z.: A general maximum principle for optimal control of forward–backward stochastic systems. Automatica 49(5), 1473–1480 (2013)
Yong, J.: Optimality variational principle for controlled forward–backward stochastic differential equations with mixed initial-terminal conditions. SIAM J. Control Optim. 48(6), 4119–4156 (2010)
Hu, M.: Stochastic global maximum principle for optimization with recursive utilities. Probab. Uncertain. Quant. Risk 2(1), 1–20 (2017)
Bell, D.J., Jacobson, D.H.: Singular Optimal Control Problems. Mathematics in Science and Engineering, vol. 117. Elsevier, London (1975)
Gabasov, R., Kirillova, F.: High order necessary conditions for optimality. SIAM J. Control 10(1), 127–168 (1972)
Kazemi-Dehkordi, M.: Necessary conditions for optimality of singular controls. J. Optim. Theory Appl. 43(4), 629–637 (1984)
Krener, A.J.: The high order maximal principle and its application to singular extremals. SIAM J. Control Optim. 15(2), 256–293 (1977)
Tang, S.: A second-order maximum principle for singular optimal stochastic controls. Discrete Contin. Dyn. Syst. Ser. B 14, 1581–1599 (2010)
Bonnans, J.F., Silva, F.J.: First and second order necessary conditions for stochastic optimal control problems. Appl. Math. Optim. 65(3), 403–439 (2012)
Zhang, H., Zhang, X.: Pointwise second-order necessary conditions for stochastic optimal controls, part I: the case of convex control constraint. SIAM J. Control Optim. 53(4), 2267–2296 (2015)
Zhang, H., Zhang, X.: Pointwise second-order necessary conditions for stochastic optimal controls, part II: the general case. SIAM J. Control Optim. 55(5), 2841–2875 (2017)
Frankowska, H., Zhang, H., Zhang, X.: First and second order necessary conditions for stochastic optimal controls. J. Differ. Equ. 262(6), 3689–3736 (2017)
Mou, L., Yong, J.: A variational formula for stochastic controls and some applications. Pure Appl. Math. Q. 3(2), 539–567 (2007)
Li, X., Yao, Y.: Maximum principle of distributed parameter systems with time lags. In: Kappel, F., Kunisch, K., Schappacher, W. (eds.) Distributed Parameter Systems, pp. 410–427. Springer, Berlin (1985)
Evans, L.C.: Partial Differential Equations. American Mathematical Society, Providence (2010)
Stein, E.M., Shakarchi, R.: Real Analysis. Princeton Lectures in Analysis III. Princeton University Press, Princeton (2005)
Framstad, N.C., Øksendal, B., Sulem, A.: Sufficient stochastic maximum principle for the optimal control of jump diffusions and applications to finance. J. Optim. Theory Appl. 121(1), 77–98 (2004)
Andersson, D., Djehiche, B.: A maximum principle for SDEs of mean-field type. Appl. Math. Optim. 63(3), 341–356 (2011)
Acknowledgements
The first author gratefully acknowledges financial support from Région Pays de la Loire through the Grant PANORisk. The second author was supported by the Natural Science Foundation of Zhejiang Province for Distinguished Young Scholar (No. LR15A010001) and the National Natural Science Foundation of China (Nos. 11871211, 11471079). Both authors would like to thank the editors and anonymous reviewers for their valuable comments and suggestions that helped improve the quality of this paper.
Author information
Authors and Affiliations
Corresponding author
Additional information
Nikolai Osmolovskii.
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Appendix: Proof for the Claim in Sect. 4.2
Appendix: Proof for the Claim in Sect. 4.2
We give the proof for the claim that there is a null subset \(\mathcal {T}_{ij}^k \subset [0,T]\) such that, for \(t \in [0,T]/\mathcal {T}_{ij}^k\),
Note that both \(\gamma (u)(\delta G(t,Z_{ij}^{u_k}(u))+\delta b^*(t,Z_{ij}^{u_k}(u))P(u))\) and \(\delta b(s,Z_{ij}^{u_k}(s))\) are square integrable processes. We shall prove a more general result that for any two processes \(f,g \in L^{2}_{\mathbb {F}}(0,T)\), there exists a null subset \(\mathcal {T} \subset [0,T]\) such that for \(t \in [0,T]/\mathcal {T}\),
This result will prove our previous claim immediately. We divide the proof into three steps.
Step 1. We prove that, for\(f \in L^{2}_{\mathbb {F}}(0,T)\), there exists a null subset\(\mathcal {T} \subset [0,T]\)such that, for\(t \in [0,T]/\mathcal {T}\),
Let \(\zeta (t)\) be the standard mollifier (for its properties see Appendix C.4 in [25]), i.e.,
and \(\zeta _n(t)=n\zeta (nt)\). We define \(f_n(t):=f \star \zeta _n(t)\) with \(\star \) representing the convolution between functions on \(\mathbb {R}\). Then, we see that \(f_n\) has continuous trajectories almost surely. Moreover, if we define a function g on [0, T] as \(g_n(t):=\mathbb {E}[|f_n(t)-f(t)|^2]\), then \(\left\| g_n\right\| _{L^1}=\int _0^T g_n(t)\hbox {d}t \longrightarrow 0\) as n goes to infinity. Define
and \(T(f)(t)=\limsup _{r \rightarrow 0}T_r(f)(t)\). It is then sufficient to prove that the set \(E_{a}:=\{t \in [0,T]| T(f)(t) \ge 2a\}\) is a null set for any \(a>0\). One can easily obtain that
for any n, with \(Mg_n\) being the Hardy–Littlewood maximal function of \(g_n\). Since \(f_n\) has continuous paths, we have \(\lim _{r \rightarrow 0} T_r(f_n)(t) =0\) for any t. Then,
which implies that
From Hardy–Littlewood maximal inequality (see [26]),
where \(C_1\) is the constant from the inequality. Using Chebychev’s inequality, we also obtain
Letting n go to infinity, we show that \(|E_a|=0\).
Step 2. There exists a null subset\(\mathcal {T} \subset [0,T]\)such that, for\(t \in [0,T]/\mathcal {T}\),
Note that
Then,
Using Hölder inequality, we have
and
From Step 1, there is a null subset \(\mathcal {T}^{\prime }\) such that, for \(t \in [0,T]/\mathcal {T}^{\prime }\), \(\frac{1}{r}\int _{t-r\beta }^{t+r\alpha } \mathbb {E}[|f(s)-f(t)|^2]^{\frac{1}{2}} \hbox {d}s \longrightarrow 0\) as r goes to 0. Since \(\mathbb {E}[|g(u)|^2]\) is Lebesgue integral on [0, T], there is a null set \(\mathcal {T}^{\prime \prime } \subset [0,T]\) such that, for \(t \in [0,T]/\mathcal {T}^{\prime \prime }\),
Define \(\mathcal {T}_1:=\mathcal {T}^{\prime } \cup \mathcal {T}^{\prime \prime }\). It is a null subset, and for \(t \in [0,T]/\mathcal {T}_1\), we have
Similarly, there is another null set \(\mathcal {T}_2\) such that, for \(t \in [0,T]/\mathcal {T}_2\),
Let \(\mathcal {T}:=\mathcal T_1 \cup \mathcal T_2\). This is the desired subset.
Step 3. Now we shall prove the final result.
Note that
Thus,
From Step 2, we know that there is a null subset \(\mathcal {T}\) such that, for \(t \in [0,T]/\mathcal {T}\), the above term tends to 0, which implies that
Thus, we finish the proof. \(\square \)
Rights and permissions
About this article
Cite this article
Dong, Y., Meng, Q. Second-Order Necessary Conditions for Optimal Control with Recursive Utilities. J Optim Theory Appl 182, 494–524 (2019). https://doi.org/10.1007/s10957-019-01518-7
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10957-019-01518-7