Deterministic control of SDEs with stochastic drift and multiplicative noise: a variational approach. (English) Zbl 1512.49024
Summary: We consider a linear stochastic differential equation with stochastic drift and multiplicative noise. We study the problem of approximating its solution with the process that solves the equation where the possibly stochastic drift is replaced by a deterministic function. To do this, we use a combination of deterministic Pontryagin’s maximum principle approach and direct methods of calculus of variations. We find necessary and sufficient conditions for a function \(u \in L^1(0, T)\) to be a minimizer of a certain cost functional. To overcome the problem of the existence of such minimizer, we also consider suitable families of penalized coercive cost functionals. Finally, we consider the important example of the quadratic cost functional, showing that the expected value of the drift component is not always the best choice in the mean squared error approximation.
MSC:
| 49J55 | Existence of optimal solutions to problems involving randomness |
| 60H10 | Stochastic ordinary differential equations (aspects of stochastic analysis) |
Software:
MatlabReferences:
| [1] | Andersson, D.; Djehiche, B., A maximum principle for SDEs of mean-field type, Appl. Math. Optim., 63, 341-356 (2011) · Zbl 1215.49034 · doi:10.1007/s00245-010-9123-8 |
| [2] | Arnold, L., Random Dynamical Systems. Monographs in Mathematics (1998), New York: Springer, New York · Zbl 0906.34001 · doi:10.1007/978-3-662-12878-7 |
| [3] | Ascione, G.; Carfora, MF; Pirozzi, E., A stochastic model for interacting neurons in the olfactory bulb, Biosystems, 185, 104030 (2019) · doi:10.1016/j.biosystems.2019.104030 |
| [4] | Ascione, G.; D’Onofrio, G.; Kostal, L.; Pirozzi, E., An optimal Gauss-Markov approximation for a process with stochastic drift and applications, Stochastic Process. Appl., 130, 11, 6481-6514 (2020) · Zbl 1454.60074 · doi:10.1016/j.spa.2020.05.018 |
| [5] | Asmussen, S.; Glynn, PW, Stochastic Simulation: Algorithms and Analysis (2007), New York: Springer, New York · Zbl 1126.65001 · doi:10.1007/978-0-387-69033-9 |
| [6] | Atkinson, KE, The Numerical Solution of Integral Equations of the Second Kind (1997), Cambridge: Cambridge University Press, Cambridge · Zbl 0899.65077 · doi:10.1017/CBO9780511626340 |
| [7] | Bauermann, J.; Lindner, B., Multiplicative noise is beneficial for the transmission of sensory signals in simple neuron models, Biosystems, 178, 25-31 (2019) · doi:10.1016/j.biosystems.2019.02.002 |
| [8] | Bayraktar, E.; Cosso, A.; Pham, H., Randomized dynamic programming principle and Feynman-Kac representation for optimal control of McKean-Vlasov dynamics, Trans. Am. Math. Soc., 370, 3, 2115-2160 (2018) · Zbl 1381.93102 · doi:10.1090/tran/7118 |
| [9] | Bellman, R., Dynamic programming and stochastic control processes, Inf. Control, 1, 3, 228-239 (1958) · Zbl 0085.34901 · doi:10.1016/S0019-9958(58)80003-0 |
| [10] | Berger, MS, Nonlinearity and Functional Analysis: Lectures on Nonlinear Problems in Mathematical Analysis (1977), Cambridge: Academic Press, Cambridge · Zbl 0368.47001 |
| [11] | Bhattacharya, R.; Majumdar, M., Random Dynamical Systems: Theory and Applications (2007), Cambridge: Cambridge University Press, Cambridge · Zbl 1114.37027 · doi:10.1017/CBO9780511618628 |
| [12] | Bonnans, JF; Silva, FJ, First and second order necessary conditions for stochastic optimal control problems, Appl. Math. Optim., 65, 403-439 (2012) · Zbl 1244.49045 · doi:10.1007/s00245-012-9162-4 |
| [13] | Brezis, H., Functional Analysis, Sobolev Spaces and Partial Differential Equations (2010), Berlin: Springer, Berlin · Zbl 1218.46002 |
| [14] | Cordoni, F.; Di Persio, L., A maximum principle for a stochastic control problem with multiple random terminal times, Math. Eng., 2, 557 (2020) · Zbl 1487.49033 · doi:10.3934/mine.2020025 |
| [15] | D’Onofrio, G.; Lansky, P.; Pirozzi, E., On two diffusion neuronal models with multiplicative noise: The mean first-passage time properties, Chaos Interdiscipl. J. Nonlinear Sci., 28, 4, 043103 (2018) · Zbl 1390.92028 · doi:10.1063/1.5009574 |
| [16] | Dacorogna, B., Direct Methods in the Calculus of Variations (2007), New York: Springer, New York |
| [17] | Davis, MHA; Burstein, G., A deterministic approach to stochastic optimal control with application to anticipative control, Stochast. Stochast. Rep., 40, 3-4, 203-256 (1992) · Zbl 0774.93085 · doi:10.1080/17442509208833790 |
| [18] | De Giorgi, E., Semicontinuity Theorems in the Calculus of Variations (2008), Berlin: Quaderni dell’Accademia Pontaniana, Accademia Pontaniana, Berlin · Zbl 1198.49002 |
| [19] | Evans, LC; Gariepy, RF, Measure Theory and Fine Properties of Functions (2015), Cambridge: CRC Press, Cambridge · Zbl 1310.28001 · doi:10.1201/b18333 |
| [20] | Faugeras, O., Soret, E., Tanré, E.: Asymptotic behaviour of a network of neurons with random linear interactions. Preprint at http://arxiv.org/abs/Probability (2019) |
| [21] | Flandoli, F., Tonello, E.: An Introduction to Random Dynamical Systems for Climate (2019). https://courseclimath19.sciencesconf.org/data/pages/RDS_4_Climate_Flandoli_v2.pdf. Accessed 07 Oct 2021 |
| [22] | Fuhrman, M.; Orrieri, C., Stochastic maximum principle for optimal control of a class of nonlinear spdes with dissipative drift, SIAM J. Control. Optim., 54, 341-371 (2016) · Zbl 1345.49036 · doi:10.1137/15M1012888 |
| [23] | Fuhrman, M., Hu, Y., Tessitore, G.: Stochastic maximum principle for optimal control of SPDEs. Preprint at http://arXiv.org/abs/1302.0286 (2013) |
| [24] | Grazieschi, P.; Leocata, M.; Mascart, C.; Chevallier, J.; Delarue, F.; Tanré, E., Network of interacting neurons with random synaptic weights, ESAIM ProcS, 65, 445-475 (2019) · Zbl 1418.92006 · doi:10.1051/proc/201965445 |
| [25] | Johnson, P.; Pedersen, J.; Peskir, G.; Zucca, C., Detecting the presence of a random drift in Brownian motion, Stochast. Process. Appl. (2021) · Zbl 1494.60044 · doi:10.1016/j.spa.2021.05.006 |
| [26] | Kalman, RE; Bellman, R., The theory of optimal control and the calculus of variations, Mathematical Optimization Techniques, 309-332 (2021), Berkeley: University of California Press, Berkeley |
| [27] | Kazamaki, N., Continuous Exponential Martingales and BMO (2006), Berlin: Springer, Berlin |
| [28] | Lánskỳ, P.; Lanska, V., Diffusion approximation of the neuronal model with synaptic reversal potentials, Biol. Cybern., 56, 1, 19-26 (1987) · Zbl 0618.92005 · doi:10.1007/BF00333064 |
| [29] | Lewis, A.D.: The Maximum Principle of Pontryagin in Control and in Optimal Control. Handouts for the Course Taught at the Universitat Politecnica de Catalunya (2006) |
| [30] | Lions, PL, Optimal control of diffusion processes and Hamilton-Jacobi-Bellman equations part 2: viscosity solutions and uniqueness, Commun. Part. Differ. Equ., 8, 11, 1229-1276 (1983) · Zbl 0716.49023 · doi:10.1080/03605308308820301 |
| [31] | Menoukeu-Pamen, O., Tangpi, L.: Maximum principle for stochastic control of SDEs with measurable drifts. Preprint at http://arxiv.org/abs/2101.06205 (2021) |
| [32] | Meyer, PA, Probability and Potentials (1966), Waltham: Blaisdell Publishing Company, Waltham · Zbl 0138.10401 |
| [33] | Øksendal, B., Stochastic Differential Equations: An Introduction with Applications (2013), New York: Springer, New York |
| [34] | Øksendal, B., Sulem, A.: A maximum principle for optimal control of stochastic systems with delay, with applications to finance. In: Optimal Control and Partial Differential Equations (Paris, 4 December 2000), pp. 64-79 (2001) · Zbl 1054.93531 |
| [35] | Orrieri, C., A stochastic maximum principle with dissipativity conditions, Discret. Contin. Dyn. Syst., 35, 11, 5499-5519 (2015) · Zbl 1332.93381 · doi:10.3934/dcds.2015.35.5499 |
| [36] | Pardoux, E.; Peng, S., Adapted solution of a backward stochastic differential equation, Syst. Control Lett., 14, 1, 55-61 (1990) · Zbl 0692.93064 · doi:10.1016/0167-6911(90)90082-6 |
| [37] | Peng, S., A general stochastic maximum principle for optimal control problems, SIAM J. Control. Optim., 28, 966-979 (1990) · Zbl 0712.93067 · doi:10.1137/0328054 |
| [38] | Pham, H., On some recent aspects of stochastic control and their applications, Probab. Surv., 2, 506-549 (2005) · Zbl 1189.93146 · doi:10.1214/154957805100000195 |
| [39] | Pick, L.; Kufner, A.; John, O.; Fucík, S., Function Spaces (2012), Berlin: Walter de Gruyter, Berlin · Zbl 1275.46002 · doi:10.1515/9783110250428 |
| [40] | Revuz, D.; Yor, M., Continuous Martingales and Brownian Motion (2013), New York: Springer, New York |
| [41] | Sacerdote, L., Giraudo, M.T.: Stochastic integrate and fire models: a review on mathematical methods and their applications. In: Stochastic Biomathematical Models, pp. 99-148 (2013) · Zbl 1390.92032 |
| [42] | Stannat, W.; Wessels, L., Deterministic control of stochastic reaction-diffusion equations, Evol. Equ. Control Theory (2020) · Zbl 1481.93054 · doi:10.3934/eect.2020087 |
| [43] | Štecha, J.; Rathouský, J., Stochastic maximum principle, IFAC Proc., 44, 1, 4714-4720 (2011) · doi:10.3182/20110828-6-IT-1002.01501 |
| [44] | The Mathworks, Inc., Natick, Massachusetts: MATLAB version 9.10.0.1710957 (R2021a) Update 4 (2021) |
| [45] | Tonelli, L.: Fondamenti di Calcolo delle Variazioni I. Zanichelli (1921) |
| [46] | Whitt, W., Stochastic-Process Limits: An Introduction to Stochastic-Process Limits and Their Application to Queues (2002), New York: Springer, New York · Zbl 0993.60001 · doi:10.1007/b97479 |
| [47] | Wing, GM, A Primer on Integral Equations of the First Kind: The Problem of Deconvolution and Unfolding (1991), Philadelphia: SIAM, Philadelphia · Zbl 0753.45001 · doi:10.1137/1.9781611971675 |
| [48] | Wong, E.; Zakai, M., On the relation between ordinary and stochastic differential equations, Int. J. Eng. Sci., 3, 2, 213-229 (1965) · Zbl 0131.16401 · doi:10.1016/0020-7225(65)90045-5 |
| [49] | Yong, J.; Zhou, XY, Stochastic Controls: Hamiltonian Systems and HJB Equations (1999), New York: Springer, New York · Zbl 0943.93002 · doi:10.1007/978-1-4612-1466-3 |
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. In some cases that data have been complemented/enhanced by data from zbMATH Open. This attempts to reflect the references listed in the original paper as accurately as possible without claiming completeness or a perfect matching.
