×

Semi-uniform Feller stochastic kernels. (English) Zbl 1542.60005

Let \(\mathbb{S}_1\) and \(\mathbb{S}_2\) be metric spaces and let \(\Psi(\cdot|\cdot)\) be a stochastic kernel (also known as a transition probability); \(\psi\) is said to be weakly continuous if \(\psi(\cdot| s^{(n)})\) converges weakly to \(\psi(\cdot| s)\). Let \(\mathbb{S}_1\), \(\mathbb{S}_2\) and \(\mathbb{S}_3\) be Borel subsets of Polish spaces; if \(\psi(A,B| s_3):=\psi(A\times B|s_3)\), the stochastic kernel \(\psi\) is said to be semi-uniform Feller if, for every sequence \(s_3^{(n)}\) that converges to \(s\) in \(\mathbb{S}_3\) one has, for every bounded continuous function \(f\) on \(\mathbb{S}_1\) \[ \lim_{n\to\infty}\,\sup_{B\in\mathcal{B}(\mathbb{S}_2)}\,\left|\int_{\mathbb{S}_1}f(s_1)\psi(ds_1,B| s_3^{(n)})- \int_{\mathbb{S}_1}f(s_1)\psi(ds_1,B| s_3)\right|=0\,. \] The authors give different equivalent definitions of this property and show its preservation under integration; they also give a motivation from the theory of Markov decision processes with incomplete information.

MSC:

60B10 Convergence of probability measures
60J05 Discrete-time Markov processes on general state spaces
90C40 Markov and semi-Markov decision processes

References:

[1] Aoki, M., Optimal control of partially observable Markovian systems, J. Frankl. Inst., 280, 5, 367-386 (1965) · Zbl 0142.14403 · doi:10.1016/0016-0032(65)90528-4
[2] Åström, KJ, Optimal control of Markov processes with incomplete state information, J. Math. Anal. Appl., 10, 174-205 (1965) · Zbl 0137.35803 · doi:10.1016/0022-247X(65)90154-X
[3] Bertsekas, DP; Shreve, SE, Stochastic Optimal Control: The Discrete-Time Case (1978), New York: Academic Press, New York · Zbl 0471.93002
[4] Bogachev, VI, Measure Theory (2007), Berlin: Springer-Verlag, Berlin · Zbl 1120.28001 · doi:10.1007/978-3-540-34514-5
[5] Dynkin, EB, Controlled random sequences, Theory Probab. Appl., 10, 1, 1-14 (1965) · Zbl 0137.35801 · doi:10.1137/1110001
[6] Dynkin, EB; Yushkevich, AA, Controlled Markov Processes (1979), New York: Springer-Verlag, New York · Zbl 0426.60063 · doi:10.1007/978-1-4615-6746-2
[7] Feinberg, E.A., Kasyanov, P.O.: Equivalent conditions for weak continuity of nonlinear filters. arXiv:2207.07544 (2022) · Zbl 1516.93265
[8] Feinberg, EA; Kasyanov, PO; Liang, Y., Fatou’s lemma in its classical form and Lebesgue’s convergence theorems for varying measures with applications to Markov decision processes, Theory Probab. Appl., 65, 2, 270-291 (2020) · Zbl 1480.28005 · doi:10.1137/S0040585X97T989945
[9] Feinberg, E.A., Kasyanov, P.O., Royset, J.O.: Epi-convergence of expectation functions under varying measures and integrands. J. Convex Anal. 30(2). arXiv:2208.03805 (2023) (to appear) · Zbl 1522.90048
[10] Feinberg, EA; Kasyanov, PO; Zgurovsky, MZ, Convergence of probability measures and Markov decision models with incomplete information, Proc. Steklov Inst. Math., 287, 1, 96-117 (2014) · Zbl 1327.60019 · doi:10.1134/S0081543814080069
[11] Feinberg, EA; Kasyanov, PO; Zgurovsky, MZ, Uniform Fatou’s lemma, J. Math. Anal. Appl., 444, 1, 550-567 (2016) · Zbl 1355.28003 · doi:10.1016/j.jmaa.2016.06.044
[12] Feinberg, EA; Kasyanov, PO; Zgurovsky, MZ, Partially observable total-cost Markov decision processes with weakly continuous transition probabilities, Math. Oper. Res., 41, 2, 656-681 (2016) · Zbl 1338.90445 · doi:10.1287/moor.2015.0746
[13] Feinberg, EA; Kasyanov, PO; Zgurovsky, MZ, Markov decision processes with incomplete information and semi-uniform Feller transition probabilities, SIAM J. Control. Optim., 60, 2488-2513 (2022) · Zbl 1498.90245 · doi:10.1137/21M1442152
[14] Feinberg, EA; Kasyanov, PO; Zgurovsky, MZ; Piunovskiy, A.; Zhang, Y., Average cost Markov decision processes with semi-uniform Feller transition probabilities, Modern Trends in Controlled Stochastic Processes: Theory and Applications, 1-18 (2021), Cham: Springer Nature, Cham · Zbl 1478.90141
[15] Hernández-Lerma, O., Adaptive Markov Control Processes (1989), New York: Springer-Verlag, New York · Zbl 0698.90053 · doi:10.1007/978-1-4419-8714-3
[16] Kara, AD; Saldi, N.; Yüksel, S., Weak Feller property of non-linear filters, Syst. Control Lett., 134, 104512 (2019) · Zbl 1428.93109 · doi:10.1016/j.sysconle.2019.104512
[17] Ma, L.: Sequential convergence on the space of Borel measures. arXiv:2102.05840 (2021)
[18] Papanicolaou, GC; Rosenblatt, M., Asymptotic analysis of stochastic equations, Studies in Probability Theory, 111-179 (1978), Washington, DC: Mathematical Association of America, Washington, DC · Zbl 0443.60049
[19] Parthasarathy, KR, Probability Measures on Metric Spaces (1967), New York: Academic Press, New York · Zbl 0153.19101 · doi:10.1016/B978-1-4832-0022-4.50006-5
[20] Rhenius, D., Incomplete information in Markovian decision models, Ann. Stat., 2, 6, 1327-1334 (1974) · Zbl 0294.49007 · doi:10.1214/aos/1176342886
[21] Rockafellar, RT; Wets, RJ-B, Variational Analysis (1998), Berlin: Springer, Berlin · Zbl 0888.49001 · doi:10.1007/978-3-642-02431-3
[22] Rudin, W., Principles of Mathematical Analysis (1964), New York: McGraw-Hill Inc, New York · Zbl 0148.02903
[23] Runggaldier, WJ; Stettner, L., Approximations of Discrete Time Partially Observed Control Problems (1994), Pisa: Applied Mathematics Monographs CNR, Giardini Editori, Pisa
[24] Schäl, M., On dynamic programming: compactness of the space of policies, Stoch. Process. Appl., 3, 345-364 (1975) · Zbl 0317.60025 · doi:10.1016/0304-4149(75)90031-9
[25] Shiryaev, A.N.: On the theory of decision functions and control by an observation process with incomplete data. Transactions of the Third Prague Conference on Information Theory, Statistical Decision Functions, Random Processes (Prague, 1962), pp. 657-681 (in Russian); Engl. transl. in Select. Transl. Math. Statist. Probab. 6(1966), 162-188 (1964) · Zbl 0207.46705
[26] Shiryaev, A.N.: Some new results in the theory of controlled random processes. Transactions of the Fourth Prague Conference on Information Theory, Statistical Decision Functions, Random Processes (Prague, 1965), pp. 131-201 (in Russian); Engl. transl. in Select. Transl. Math. Statist. Probab. 8(1969), 49-130 (1967)
[27] Smallwood, RD; Sondik, EJ, The optimal control of partially observable Markov processes over a finite horizon, Oper. Res., 21, 5, 1071-1088 (1973) · Zbl 0275.93059 · doi:10.1287/opre.21.5.1071
[28] Yushkevich, AA, Reduction of a controlled Markov model with incomplete data to a problem with complete information in the case of Borel state and control spaces, Theory Probab. Appl., 21, 1, 153-158 (1976) · Zbl 0357.93040 · doi:10.1137/1121014
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. In some cases that data have been complemented/enhanced by data from zbMATH Open. This attempts to reflect the references listed in the original paper as accurately as possible without claiming completeness or a perfect matching.