×

Finite-memory suboptimal design for partially observed Markov decision processes. (English) Zbl 0833.90123

Partially observed Markov decision processes are difficult to handle since either the whole past history or a sufficient statistic (usually uncountable) have to be considered. Here, a finite-stage past history is used as a suboptimal information set to construct bounds on the optimal value functions. The contraction properties (ergodicity) of the pertinent \(M\)-stage operators are investigated and proved to be favorable with increasing \(M\). On the other hand with increasing \(M\) the calculation effort grows exponentially. These effects are demonstrated by some numerical examples.

MSC:

90C40 Markov and semi-Markov decision processes
Full Text: DOI