Article Contents

2018, Volume 5, Issue 2: 143-163. Doi: 10.3934/jdg.2018009

This issue Previous Article Constrained stochastic differential games with additive structure: Average and discount payoffs Next Article Stable manifold market sequences

A risk minimization problem for finite horizon semi-Markov decision processes with loss rates

Qiuli Liu^1, , and
Xiaolong Zou^2,

1.
School of Mathematical Sciences, South China Normal University, Guangzhou 510631, China
2.
School of Economics and Statistics, Guangzhou University, Guangzhou 510006, China

* Corresponding author. Emails: liuql@m.scnu.edu.cn; zouxiaol@gzhu.edu.cn
* Corresponding author. Emails: liuql@m.scnu.edu.cn; zouxiaol@gzhu.edu.cn

Received: June 2017

Revised: July 2017

Published: March 2018

Research supported by Natural Science Foundation of Guangdong Province (Grant No.2014A030313438), Zhujiang New Star (Grant No. 201506010056) and Guangdong Province outstanding young teacher training plan (Grant No. YQ2015050)

Abstract / Introduction Full Text(HTML) Figure(5) Related Papers Cited by

Abstract

This paper deals with the risk probability for finite horizon semi-Markov decision processes with loss rates. The criterion to be minimized is the risk probability that the total loss incurred during a finite horizon exceed a loss level. For such an optimality problem, we first establish the optimality equation, and prove that the optimal value function is a unique solution to the optimality equation. We then show the existence of an optimal policy, and develop a value iteration algorithm for computing the value function and optimal policies. We also derive the approximation of the value function and the rules of iteration. Finally, a numerical example is given to illustrate our results.

Keywords:

Mathematics Subject Classification: Primary: 90C40; Secondary: 93E20.

Citation:

Full Text(HTML)

Figure 1. The function $F^*_{(n_0+1)k}(1, t, \lambda)$

Download: Full-size image PowerPoint slide

Figure 2. The function $F^*_{(n_0+1)k}(2, t, \lambda)$

Download: Full-size image PowerPoint slide

Figure 3. The function $H^aF^*_{(n_0+1)k-1}(i, 10, \lambda)$

Download: Full-size image PowerPoint slide

Figure 4. The function $H^aF^*_{(n_0+1)k-1}(i, 15, \lambda)$

Download: Full-size image PowerPoint slide

Figure 5. The function $\lambda^*(i, t)$

Download: Full-size image PowerPoint slide

Related Papers

Cited by

References

	N. Bauerle and U. Rieder, Markov Decision Processes with Application to Finance, Universitext, Springer, Heidelberg, 2011.
	M. Bouakiz and Y. Kebir , Target-level criterion in Markov decision processes, Journal of Optimization Theory and Applications, 86 (1995) , 1-15. doi: 10.1007/BF02193458.
	M. K. Ghosh and S. Subhamay , Non-stationary semi-Markov secision processes on a finite horizon, Stochastic Analysis and Applications, 31 (2013) , 183-190. doi: 10.1080/07362994.2013.741405.
	X. P. Guo and O. Hernández-Lerma, Continuous-Time Markov Decision Processes: Theory and Applications, Springer-Verlag, Berlin, 2009.
	X. P. Guo and J. Yang , A new condition and approach for zero-sum stochastic games with average payoffs, Stochastic Analysis and Applications, 26 (2008) , 537-561. doi: 10.1080/07362990802007095.
	X. P. Guo , P. Shi and W. P. Zhu , Strong average optimality for controlled nonhomogeneous Markov chains, Stochastic Analysis and Applications, 19 (2001) , 115-134. doi: 10.1081/SAP-100001186.
	O. Hernández-Lerma and J. B. Lasserre, Discrete-time Markov Control Processes, Basic optimality criteria, Springer-Verlag, New York, 1996.
	O. Hernández-Lerma and J. B. Lasserre, Further Topics on Discrete-Time Markov Control Processes, Springer-Verlag, New York, 1999.
	Y. H. Huang and X. P. Guo , Optimal risk probability for first passage models in semi-Markov decision processes, Journal Mathematical Analysis Applications, 359 (2009) , 404-420. doi: 10.1016/j.jmaa.2009.05.058.
	Y. H. Huang , X. P. Guo and X. Y. Song , Performance analysis for controlled semi-Markov systems with application to maintenance, Journal of Optimization Theory and Applications, 150 (2011) , 395-415. doi: 10.1007/s10957-011-9813-7.
	Y. H. Huang and X. P. Guo , Finite horizon semi-Markov decision processes with application to maintenance systems, European Journal Operations Research, 212 (2011) , 131-140. doi: 10.1016/j.ejor.2011.01.027.
	Y. H. Huang , X. P. Guo and Z. F. Li , Minimum risk probability for finite horizon semi-Markov decision processes, Journal Mathematical Analysis Applications, 402 (2013) , 378-391. doi: 10.1016/j.jmaa.2013.01.021.
	Y. H. Huang and X. P. Guo , Mean-variance problems for finite horizon semi-Markov decision processes, Applications Mathematical Optimization, 72 (2015) , 233-259. doi: 10.1007/s00245-014-9278-9.
	N. Limnios and G. Oprisan, Semi-Markov Processes and Reliability, Birkhäuser Boston, Inc., Boston, MA, 2001.
	J. Y. Liu and S. M. Huang , Markov decision processes with distribution function criterion of first-passage time, Applications Mathematical Optimization, 43 (2001) , 187-201. doi: 10.1007/s00245-001-0007-9.
	P. M. Madhani , Rebalancing fixed and variable pay in a sales organization: A business cycle perspective, Compensation Benefits Review, 42 (2010) , 179-189. doi: 10.1177/0886368709359668.
	J. W. Mamer , Successive approximations for finite horizon semi-Markov decision processes with application to asset liquidation, Oper. Res., 34 (1986) , 638-644. doi: 10.1287/opre.34.4.638.
	Y. Ohtsubo , Minimizing risk models in stochastic shortest path problems, Mathematical Methods of Operations Research, 57 (2003) , 79-88. doi: 10.1007/s001860200246.
	Y. Ohtsubo , Optimal threshold probability in undiscounted Markov decision processes with a target set, Appl. Math. Comput., 149 (2004) , 519-532. doi: 10.1016/S0096-3003(03)00158-9.
	M. L. Puterman, Markov Decision Processes: Discrete Stochastic Dynamic Programming, John Wiley & Sons, Inc., New York, 1994.
	C. Ruhm , Are recessions good for your health?, Quarterly Journal of Economics, 115 (2000) , 617-650. doi: 10.3386/w5570.
	M. Sakaguchi and Y. Ohtsubo , Markov decision processes associated with two threshold probability criteria, Journal Control Theory Applications, 11 (2013) , 548-557. doi: 10.1007/s11768-013-2194-8.
	Q. D. Wei and X. P. Guo , New average optimality conditions for semi-Markov decision processes in Borel spaces, Journal of Optimization Theory and Applications, 153 (2012) , 709-732. doi: 10.1007/s10957-012-9986-8.
	D. J. White , Minimising a threshold probability in discounted Markov decision processes, J. Math. Anal. Appl., 173 (1993) , 634-646. doi: 10.1006/jmaa.1993.1093.
	Y. H. Wu , Bounds for the ruin probability under a Markovian modulated risk model, Communications in statistics Stochastic Models, 15 (1999) , 125-136. doi: 10.1080/15326349908807529.
	S. X. Yu , Y. L. Lin and P. F. Yan , Optimization models for the first arrival target distribution function in discrete time, J. Math. Anal. Appl., 225 (1998) , 193-223. doi: 10.1006/jmaa.1998.6015.