×

About stability of risk-seeking optimal stopping. (English) Zbl 1300.60059

Summary: We offer the quantitative estimation of stability of risk-sensitive cost optimization in the problem of optimal stopping of Markov chains on a Borel space \(X\). It is supposed that the transition probability \(p(\cdot |x)\), \(x\in X\), is approximated by the transition probability \(\widetilde{p}(\cdot |x)\), \(x\in X\), and that the stopping rule \(\widetilde{f}_{\ast}\), which is optimal for the process with the transition probability \(\widetilde{p}\), is applied to the process with the transition probability \(p\). We give an upper bound (expressed in term of the total variation distance: \(\sup_{x\in X}\|p(\cdot |x)-\widetilde{p}(\cdot |x)\|\)) for the additional cost paid for using the rule \(\widetilde{f}_{\ast}\) instead of the (unknown) stopping rule \(f_{\ast}\) optimal for \(p\).

MSC:

60G40 Stopping times; optimal stopping problems; gambling theory

References:

[1] Avila-Godoy, G., Fernández-Gaucherand, E.: Controlled Markov chains with exponential risk-sensitive criteria: modularity, structured policies and applications. Decision and Control 1998. Proc. 37th IEEE Conference. Vol. 1, IEEE, pp. 778-783.
[2] Bäuerle, N., Rieder, U.: Markov Decision Processes with Applications to Finance. Springer-Verlag, Berlin 2011. · Zbl 1236.90004 · doi:10.1007/978-3-642-18324-9
[3] Borkar, V. S., Meyn, S. P.: Risk-sensitive optimal control for Markov decision processes with monotone cost. Math. Oper. Res. 27 (2002), 192-209. · Zbl 1082.90577 · doi:10.1287/moor.27.1.192.334
[4] Cavazos-Cadena, R.: Optimality equations and inequalities in a class of risk-sensitive average cost Markov decision chains. Math. Methods Oper. Res. 71 (2010), 47-84. · Zbl 1189.93144 · doi:10.1007/s00186-009-0285-6
[5] Cavazos-Cadena, R., Fernández-Gaucherand, E.: Controlled Markov chains with risk-sensitive criteria: Average costs, optimality equations, and optimal solutions. Math. Methods Oper. Res. 49 (1999), 299-324. · Zbl 0953.93077
[6] Cavazos-Cadena, R., Montes-de-Oca, R.: Optimal stationary policies in risk-sensitive dynamic programs with finite state space and nonegative rewards. Appl. Math. 27 (2000), 167-185. · Zbl 1006.93070
[7] Dijk, N. M. Van, Sladký, K.: Error bounds for nonnegative dynamic models. J. Optim. Theory Appl. 101 (1999), 449-474. · Zbl 0946.90106 · doi:10.1023/A:1021749829267
[8] Devroye, L., Györfy, L.: Nonparametric Density Estimation: The \(L_1\) View. John Wiley, New York 1986.
[9] Dynkin, E. B., Yushkevich, A. A.: Controlled Markov Processes. Springer Verlag, New York 1979. · Zbl 0073.34801
[10] Gordienko, E. I., Yushkevich, A. A.: Stability estimates in the problem of average optimal switching of a Markov chain. Math. Methods Oper. Res. 57 (2003), 345-365. · Zbl 1116.90401 · doi:10.1007/s001860200258
[11] Gordienko, E. I., Lemus-Rodríguez, E., Montes-de-Oca, R.: Average cost Markov control processes: stability with respect to the Kantorovich metric. Math. Methods Oper. Res. 70 (2009), 13-33. · Zbl 1176.60062 · doi:10.1007/s00186-008-0229-6
[12] Gordienko, E. I., Salem, F.: Robustness inequalities for Markov control processes with unbounded costs. Syst. Control Lett. 33 (1998), 125-130. · Zbl 0902.93068 · doi:10.1016/S0167-6911(97)00077-7
[13] Hernández-Lerma, O., Lasserre, J. B.: Further Topics on Discrete-time Markov Control Processes. Springer-Verlag, New York 1999. · Zbl 0928.93002
[14] Jaśkiewicz, A.: Average optimality for risk-sensitive control with general state space. Ann. Appl. Probab. 17 (2007), 654-675. · Zbl 1128.93056 · doi:10.1214/105051606000000790
[15] Kartashov, N. V.: Strong Stable Markov Chains. VSP, Utrecht 1996. · Zbl 0874.60082
[16] Marcus, S. I., Fernández-Gaucherand, E., Hernández-Hernández, D. E., Coraluppi, S., Fard, P.: Risk sensitive Markov decision processes. Progress in System and Control Theory 22 (1997), 263-280. · Zbl 1065.90543
[17] Masi, G. B. Di, Stettner, L.: Infinite horizon risk sensitive control of discrete time Markov processes with small risk. Systems Control Lett. 40 (2000), 15-20. · Zbl 0977.93083 · doi:10.1016/S0167-6911(99)00118-8
[18] Meyn, S. P., Tweedie, R. L.: Markov Chains and Stochastic Stability. Springer-Verlag, London 1993. · Zbl 1165.60001 · doi:10.1017/CBO9780511626630
[19] Montes-de-Oca, R., Salem-Silva, F.: Estimates for perturbations of an average Markov decision processes with a minimal state and upper bounded stochastically ordered Markov chains. Kybernetika 41 (2005), 757-772. · Zbl 1249.90313
[20] Muciek, B. K.: Optimal stopping of risk processes: model with interest rates. J. Appl. Probab. 39 (2002), 261-270. · Zbl 1011.62111 · doi:10.1239/jap/1025131424
[21] Shiryaev, A. N.: Optimal Stopping Rules. Springer-Verlag, New York 1978. · Zbl 1138.60008 · doi:10.1007/978-3-540-74011-7
[22] Shiryaev, A. N.: Essential of Stochastic Finance. Facts, Models, Theory. World Scientific Publishing Co., Inc., River Edge, N. J. 1999.
[23] Sladký, K.: Bounds on discrete dynamic programming recursions I. Kybernetika 16 (1980), 526-547. · Zbl 0454.90085
[24] Zaitseva, E.: Stability estimating in optimal stopping problem. Kybernetika 44 (2008), 400-415. · Zbl 1154.60326
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. In some cases that data have been complemented/enhanced by data from zbMATH Open. This attempts to reflect the references listed in the original paper as accurately as possible without claiming completeness or a perfect matching.