A numerical method for ergodic optimal control of switching diffusions with reflection. (English) Zbl 1537.93788
Summary: The ergodic or long run average cost control problem for diffusions is one of the classical problems of stochastic control that still eludes a completely satisfactory treatment. This is certainly true for the setting in which the systems to be controlled is modeled by the solution of a switching stochastic differential equation with reflection (SSDER). In this paper, we advance in this rather difficult problem by setting forth preliminary results for the unidimensional case. Besides carving out a numerical method, our treatment of the ergodic control in this scenario straddles issues of existence and uniqueness of solution of the SSDER and a verification theorem for the associated HJB equation. We conclude by illustrating the effectiveness of the method considering the control of energy consumption in a large parallel processing computer system composed of one queue and several processing stations.
MSC:
| 93E20 | Optimal stochastic control |
| 65K10 | Numerical optimization and variational techniques |
| 60H30 | Applications of stochastic analysis (to PDEs, etc.) |
| 49M25 | Discrete approximations in optimal control |
References:
| [1] | Araposthatis, A.; Borkar, V. S.; Fernandez-Gaucherand, E.; Ghosh, M. K.; Marcus, S. I., Discrete-time controlled markov processes with average cost criterion: A survey, SIAM Journal on Control and Optimization, 31, 282-344, 1993 · Zbl 0770.93064 |
| [2] | Araposthatis, A.; Borkar, V. S.; Ghosh, M. K., Ergodic control of diffusion processes. Encyclopedia of mathematics and its applications, 2012, Cambridge University Press · Zbl 1236.93137 |
| [3] | Bhatt, A. G.; Borkar, V. S., Occupation measures for controlled markov processes: Characterization and optimality, The Annals of Probability, 24, 1531-1562, 1996 · Zbl 0863.93086 |
| [4] | Billingsley, P., Convergence of probability measures, 1999, John Wiley & Sons, John Wiley & Sons: John Wiley & Sons, John Wiley & Sons New York · Zbl 0944.60003 |
| [5] | Budhiraja, A., An ergodic control problem for constrained diffusion processes: Existence of optimal markov control, SIAM Journal on Control and Optimization, 42, 532-558, 2003 · Zbl 1037.93073 |
| [6] | Ethier, S. N.; Kurtz, T. G., Markov process characterization and convergence, 1986, John Wiley & Sons: John Wiley & Sons Hoboken, NJ · Zbl 0592.60049 |
| [7] | Fragoso, M. D.; Baczynski, J., Optimal control for continuous-time linear quadratic problems with infinite Markov jump parameters, SIAM Journal on Control and Optimization, 40, 270-297, 2001 · Zbl 1058.93058 |
| [8] | Fragoso, M. D.; Hemerly, E. M., Optimal control for a class of noisy linear systems with Markovian jumping parameters and quadratic cost, International Journal of Systems Science, 22, 2553-2561, 1991 · Zbl 0749.49020 |
| [9] | Harrison, J. M., Brownian motion and stochastic flow systems, 1990, Krieger Pub. Co.: Krieger Pub. Co. Malabar, Florida · Zbl 0716.60092 |
| [10] | Jacod, J.; Shiryaev, A. N., (Limit theorems for stochastic processes. Limit theorems for stochastic processes, Grundlehren der mathematischen wissenschaften, 1987, Springer-Verlag: Springer-Verlag New York) · Zbl 0635.60021 |
| [11] | Jacquet, Q., van Ackooij, W., Alasseur, C., & Gaubert, S. (2022). Ergodic control of a heterogeneous population and application to electricity pricing. In Proc. IEEE conference on decision and control (pp. 3617-3624). Cancun, Mexico. |
| [12] | Kang, W. N.; Williams, R. J., An invariance principle for semimartingale reflecting Brownian motions in domains with piecewise smooth boundaries, The Annals of Applied Probability, 17, 741-779, 2007 · Zbl 1125.60030 |
| [13] | Karatzas, I.; Shreve, S., Brownian motion and stochastic calculus, 1991, Springer: Springer New York · Zbl 0734.60060 |
| [14] | Kruk, L.; Lehoczky, J.; Ramanan, K.; Shreve, S., An explicit formula for the Skorokhod map on [0, a], The Annals of Probability, 35, 1740-1768, 2007 · Zbl 1139.60017 |
| [15] | (Kushner, H. J., Probability methods for approximations in stochastic control and for elliptic equations. Vol. 126, 1977, Academic Press: Academic Press New York) · Zbl 0547.93076 |
| [16] | Kushner, H. J., Numerical methods for stochastic control problems in continuous time, SIAM Journal on Control and Optimization, 28, 999-1048, 1990 · Zbl 0721.93087 |
| [17] | Kushner, H. J., (Weak convergence methods and singularly perturbed stochastic control and filtering problems. Weak convergence methods and singularly perturbed stochastic control and filtering problems, Systems & control: foundations & applications, 1990, Birkhäuser: Birkhäuser Boston) · Zbl 0931.93003 |
| [18] | Kushner, H. J., (Heavy traffic analysis of controlled queueing and communication networks (stochastic modelling and applied probability). Heavy traffic analysis of controlled queueing and communication networks (stochastic modelling and applied probability), Applications of mathematics, Vol. 47, 2001, Springer-Verlag: Springer-Verlag New York) · Zbl 0988.90004 |
| [19] | Kushner, H. J.; Dupuis, P. G., (Numerical methods for stochastic control problems in continuous time. Numerical methods for stochastic control problems in continuous time, Stochastic modelling and applied probability, 2001, Springer-Verlag: Springer-Verlag New York) · Zbl 0968.93005 |
| [20] | Leite, S. C.; Fragoso, M. D.; Teixeira, R. S., Switching diffusion approximations for optimal power management in parallel processing systems, Stochastic Models, 37, 367-414, 2021 · Zbl 1473.90057 |
| [21] | Leite, S. C.; Williams, R. J., A constrained Langevin approximation for chemical reaction networks, The Annals of Applied Probability, 29, 1541-1608, 2019 · Zbl 1467.60059 |
| [22] | Liu, J.; Yiu, K. F.C.; Bensoussan, A., Ergodic control for a mean reverting inventory model, Journal of Industrial and Management Optimization, 14, 857-876, 2018 · Zbl 1412.35187 |
| [23] | Menaldi, J. L.; Perthame, B.; Robin, M., Ergodic problem for optimal stochastic switching, Journal of Mathematical Analysis and Applications, 147, 512-530, 1990 · Zbl 0721.49019 |
| [24] | Miller, L. M.; Silverman, Y.; MacIver, M. A.; Murphey, T., Ergodic exploration of distributed information, IEEE Transactions in Robotics, 32, 35-52, 2016 |
| [25] | Mitrani, I., Managing performance and power consumption in a server farm, Annals of Operations Research, 202, 121-134, 2013 · Zbl 1260.90069 |
| [26] | Nguyen, D.; Yin, G., Modeling and analysis of switching diffusion systems: Past-dependent switching with a countable state space, SIAM Journal on Control and Optimization, 54, 2450-2477, 2016 · Zbl 1391.93199 |
| [27] | Puterman, M. L., Markov decision processes: discrete stochastic dynamic programming, 1994, John Wiley and Sons, John Wiley and Sons: John Wiley and Sons, John Wiley and Sons New York · Zbl 0829.90134 |
| [28] | Rogers, L. C.G.; Williams, D., Diffusions, Markov processes and martingales: volume 2, Itô calculus, 2000, Cambridge University Press: Cambridge University Press Cambridge, U.K., New York · Zbl 0977.60005 |
| [29] | Song, Q. S.; Yin, G.; Zhang, Z., Numerical methods for controlled regime-switching diffusions and regime-switching jump diffusions, Automatica, 42, 1147-1157, 2006 · Zbl 1117.93370 |
| [30] | StockBridge, R. H., Time-average control of martingale problems: A linear programming formulation, The Annals of Probability, 18, 206-217, 1990 · Zbl 0699.49019 |
| [31] | StockBridge, R. H., Time-average control of martingale problems: Existence of a stationary solution, The Annals of Probability, 18, 190-205, 1990 · Zbl 0699.49018 |
| [32] | Tanaka, H., Stochastic differential equations with reflecting boundary condition in convex regions, Hiroshima Mathematical Journal, 9, 163-177, 1979 · Zbl 0423.60055 |
| [33] | U. S. Environmental Protection Agency, Epa report on server and data center energy efficiency, 2007, ENERGY STAR Program |
| [34] | Yin, G.; Zhang, C.; Wang, L. Y., Optimal ergodic control of nonlinear stochastic systems, (Krée, P.; Wedig, W., Probabilistic methods in applied physics, 1995, Springer International Publishing), 239-269 · Zbl 0838.93074 |
| [35] | Yin, G.; Zhang, C.; Wang, L. Y., Numerical methods for controlled switching diffusions, (Lirkov, I.; Margenov, S., Large-scale scientific computing, 2018, Springer International Publishing: Springer International Publishing Cham), 33-44 · Zbl 1476.60006 |
| [36] | Yin, G.; Zhu, C., (Hybrid switching diffusions: properties and applications. Hybrid switching diffusions: properties and applications, Stochastic modelling and applied probability, 2010, Springer-Verlag: Springer-Verlag New York) · Zbl 1279.60007 |
| [37] | Zhang, C.; Yin, G.; Zhang, Q.; Wang, L. Y., Pollution control for switching diffusion models: Approximation methods and numerical results, Discrete & Continuous Dynamical Systems - B, 24, 3667, 2019 · Zbl 1478.93746 |
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. In some cases that data have been complemented/enhanced by data from zbMATH Open. This attempts to reflect the references listed in the original paper as accurately as possible without claiming completeness or a perfect matching.
