Denumerable continuous-time Markov decision processes with multiconstraints on average costs. (English) Zbl 1258.93125
Summary: This article deals with multi-constrained continuous-time Markov decision processes in a denumerable state space, with unbounded cost and transition rates. The criterion to be optimized is the long-run expected average cost, and several kinds of constraints are imposed on some associated costs. The existence of a constrained optimal policy is ensured under suitable conditions by using a martingale technique and introducing an occupation measure. Furthermore, for the uni-chain model, we transform this multi-constrained problem into an equivalent linear programming problem, then construct a constrained optimal policy from an optimal solution for the linear programming problem. Finally, we use an example of a controlled queueing system to illustrate an application of our results.
MSC:
| 93E20 | Optimal stochastic control |
| 60J05 | Discrete-time Markov processes on general state spaces |
| 90C05 | Linear programming |
References:
| [1] | DOI: 10.1080/00207729008910569 · Zbl 0729.90041 · doi:10.1080/00207729008910569 |
| [2] | DOI: 10.1007/978-1-4612-3038-0 · doi:10.1007/978-1-4612-3038-0 |
| [3] | DOI: 10.1080/00207727308920068 · Zbl 0273.60055 · doi:10.1080/00207727308920068 |
| [4] | Bertsekas DP, Stochastic Optimal Control: The Discrete-time Case (1996) |
| [5] | DOI: 10.1142/9789812562456 · doi:10.1142/9789812562456 |
| [6] | DOI: 10.1090/S0002-9947-1940-0002697-3 · JFM 66.0624.02 · doi:10.1090/S0002-9947-1940-0002697-3 |
| [7] | DOI: 10.1109/TAC.2007.899040 · Zbl 1366.90217 · doi:10.1109/TAC.2007.899040 |
| [8] | DOI: 10.1287/moor.1060.0210 · Zbl 1278.90426 · doi:10.1287/moor.1060.0210 |
| [9] | DOI: 10.1109/TAC.2002.808469 · Zbl 1364.90346 · doi:10.1109/TAC.2002.808469 |
| [10] | Guo XP, Mathematics of Operations Research 67 pp 323– (2003) |
| [11] | DOI: 10.1023/B:ACAP.0000003675.06200.45 · Zbl 1043.93067 · doi:10.1023/B:ACAP.0000003675.06200.45 |
| [12] | DOI: 10.1007/978-3-642-02547-1 · doi:10.1007/978-3-642-02547-1 |
| [13] | DOI: 10.1007/s001860000071 · Zbl 1032.90061 · doi:10.1007/s001860000071 |
| [14] | DOI: 10.1137/S0363012999361627 · Zbl 1049.90116 · doi:10.1137/S0363012999361627 |
| [15] | Hernández-Lerma O, Discrete-time Markov Control Processes (1996) · Zbl 0698.90053 |
| [16] | Hernández-Lerma O, Further Topics on Discrete-tIme Markov Control Processes (2002) |
| [17] | DOI: 10.1137/060668857 · Zbl 1165.93040 · doi:10.1137/060668857 |
| [18] | Puterman ML, Markov Decision Processes: Discrete Stochastic Dynamic Programming (1996) |
| [19] | DOI: 10.1080/00207720310001624113 · Zbl 1083.90510 · doi:10.1080/00207720310001624113 |
| [20] | DOI: 10.1007/s10959-008-0163-9 · Zbl 1147.60050 · doi:10.1007/s10959-008-0163-9 |
| [21] | DOI: 10.1007/s00186-007-0154-0 · Zbl 1143.90033 · doi:10.1007/s00186-007-0154-0 |
| [22] | DOI: 10.1016/j.jmaa.2007.06.071 · Zbl 1156.90023 · doi:10.1016/j.jmaa.2007.06.071 |
| [23] | DOI: 10.1239/jap/1214950357 · Zbl 1189.90187 · doi:10.1239/jap/1214950357 |
| [24] | Zhu QX, Abstract and Applied Analysis (2009) |
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. In some cases that data have been complemented/enhanced by data from zbMATH Open. This attempts to reflect the references listed in the original paper as accurately as possible without claiming completeness or a perfect matching.
