Abstract
In this paper we give conditions for (the existence and) several characterizations of overtaking optimal policies for a general class of controlled diffusion processes. Our characterization results are of a lexicographical type; namely, first we identify the class of so-called canonical policies, and then within this class we search for policies with some special feature—for instance, canonical policies that in addition maximize the bias.
Similar content being viewed by others
References
Arapostathis, A., Ghosh, M.K., Borkar, V.S.: Ergodic Control of Diffusion Processes (2006, in preparation)
Atsumi, H.: Neoclassical growth and the efficient program of capital accumulation. Rev. Econ. Stud. 32, 127–136 (1965)
Bensoussan, A., Frehse, J.: On Bellman equations of ergodic control in ℝn. J. Reine Angew. Math. 429, 125–160 (1992)
Bhattacharya, R.N.: On classical limit theorems for diffusions. Sankhya 44, 47–71 (1982)
Borkar, V.S., Ghosh, M.K.: Ergodic control of multidimensional diffusions II: Adaptive control. Appl. Math. Optim. 21, 191–220 (1990)
Carlson, D.A., Haurie, D.B., Leizarowitz, A.: Infinite Horizon Optimal Control: Deterministic and Stochastic Systems. Springer, Berlin (1991)
Dana, R.A., Le Van, C.: On the Bellman equation of the overtaking criterion. J. Optim. Theory Appl. 67, 587–600 (1990)
Durrett, R.: Stochastic Calculus: A Practical Introduction. CRC Press, Boca Raton (1996)
Fort, G., Roberts, G.O.: Subgeometric ergodicity of strong Markov processes. Ann. Appl. Probab. 15, 1565–1589 (2005)
Friedman, A.: Partial Differential Equations of Parabolic Type. Prentice Hall, Englewood Cliffs (1964)
Friedman, A.: Stochastic Differential Equations and Applications, vol. 1. Academic Press, New York (1975)
Ghosh, M.K., Arapostathis, A., Marcus, S.I.: Optimal control of switching diffusions to flexible manufacturing systems. SIAM J. Control Optim. 31, 1183–1204 (1993)
Ghosh, M.K., Arapostathis, A., Marcus, S.I.: Ergodic control of switching diffusions. SIAM J. Control Optim. 35, 1952–1988 (1997)
Ghosh, M.K., Marcus, S.I.: Infinite horizon controlled diffusion problems with nonstandard criteria. J. Math. Syst. Estim. Control 1, 44–69 (1991)
Glynn, P.W., Meyn, S.P.: A Lyapunov bound for solutions of the Poisson equation. Ann. Probab. 24, 916–931 (1996)
Gordienko, E., Hernández-Lerma, O.: Average cost Markov control processes with weighted norms: existence of canonical policies. Appl. Math. (Warsaw) 23, 199–218 (1995)
Guo, X.P., Hernández-Lerma, O.: Continuous-time Markov chains with discounted rewards. Acta Appl. Math. 79, 195–216 (2003)
Hashemi, S.N., Heunis, A.J.: On the Poisson equation for singular diffusions. Stochastics 77, 155–189 (2005)
Has’minskii, R.Z.: Stochastic Stability of Differential Equations. Sijthoff & Noordhoff, Rockville (1980)
Hernández-Lerma, O.: Lectures on Continuous-Time Markov Control Processes. Sociedad Matemática Mexicana, Mexico City (1994)
Hernández-Lerma, O., Lasserre, J.B.: Discrete-Time Markov Control Processes. Springer, New York (1996)
Hernández-Lerma, O., Lasserre, J.B.: Further Topics on Discrete-Time Markov Control Processes. Springer, New York (1999)
Himmelberg, C.J., Parthasarathy, T., Van Vleck, F.S.: Optimal plans for dynamic programming problems. Math. Oper. Res. 1, 390–394 (1976)
Klokov, S.A., Veretennikov, A.Y.: On subexponential mixing rate for Markov processes. Theory Probab. Appl. 49, 110–122 (2005)
Kuratowski, K.: Topology II. Academic Press, New York (1968)
Le Van, C., Dana, R.A.: Dynamic Programming in Economics. Kluwer, Dordrecht (2003)
Leizarowitz, A.: Optimal controls for diffusion in ℝd—min–max max–min formula for the minimal cost growth rate. J. Math. Anal. Appl. 149, 180–209 (1990)
Leizarowitz, A.: Controlled diffusion process on infinite horizon with the overtaking criterion. Appl. Math. Optim. 17, 61–78 (1988)
Meyn, S.P., Tweedie, R.L.: Markov Chains and Stochastic Stability. Springer, New York (1993)
Meyn, S.P., Tweedie, R.L.: Stability of Markovian processes III: Foster–Lyapunov criteria for continuous-time precesses. Adv. Appl. Probab. 25, 518–548 (1993)
Nowak, A.S., Vega-Amaya, O.: A counterexample on overtaking optimality. Math. Meth. Oper. Res. 49, 435–439 (1998)
Pardoux, E., Veretennikov, A.Y.: On the Poisson equation and diffusion approximation I. Ann. Probab. 29, 1061–1085 (2001)
Prieto-Rumeau, T., Hernandez-Lerma, O.: Bias and overtaking equilibria for zero-sum continuous-time Markov games. Math. Meth. Oper. Res. 61, 437–454 (2005)
Prieto-Rumeau, T., Hernández-Lerma, O.: Bias optimality for continuous-time controlled Markov chains. SIAM J. Control Optim. 45, 51–73 (2006)
Ramsey, F.P.: A mathematical theory of savings. Econ. J. 38, 543–559 (1928)
Schäl, M.: Conditions for optimality and for the limit of n-stage optimal policies to be optimal. Z. Wahrs. Verw. Gerb. 32, 179–196 (1975)
Tan, H., Rugh, W.J.: Nonlinear overtaking optimal control: sufficiency, stability, and approximation. IEEE Trans. Autom. Control 43, 1703–1718 (1998)
Tan, H., Rugh, W.J.: On overtaking optimal tracking for linear systems. Syst. Control Lett. 33, 63–72 (1998)
Tuominen, P., Tweedie, R.L.: The recurrence structure of general Markov processes. Proc. Lond. Math. Soc. 39, 554–576 (1979)
von Weizsäcker, C.C.: Existence of optimal programs of accumulation for an infinite horizon. Rev. Econ. Stud. 32, 85–104 (1965)
Zaslavski, A.J.: Turnpike Properties in the Calculus of Variations and Optimal Control. Springer, New York (2006)
Author information
Authors and Affiliations
Corresponding author
Additional information
This research was partially supported by CONACyT grant 45693-F. The research of the first author (HJF) was also supported by a CONACyT scholarship.
Rights and permissions
About this article
Cite this article
Jasso-Fuentes, H., Hernández-Lerma, O. Characterizations of Overtaking Optimality for Controlled Diffusion Processes. Appl Math Optim 57, 349–369 (2008). https://doi.org/10.1007/s00245-007-9025-6
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s00245-007-9025-6