This book is devoted to problems of labelled Markov processes, i.e., processes with Markov behavior which depend not only on a current position but also on some action of environment. This general model is the base for many special models which usually refer to the theory of Markov decision processes. Unlike these particular theories, the author is interested in the logical side of the model and in justifying of approximate evaluations connected with such processes. Taking into account the final aim to give a mathematical foundation for studying such processes the author spends almost half the volume of the book (6 chapters from 13) on the mathematical base of general probability theory, including series of fine results on measure theory, integration, and stochastic relations.
The proper titular subject of investigation begins from the 7th chapter where some important equivalence relations with respect to actions of environment are being studied. In the 8th chapter, some metrics and pseudometrics are studied. They are the main tools for approximate methods of evaluations described in Chapters 9 and 10. In Chapter 11 some alternative constructions of labelled Markov chains and their mutual transformations are considered. Some problems arise when one has to study labelled Markov processes in continuous time. These problems are solved in Chapter 12. In the 13th chapter some directions of investigations which are not touched in this book are discussed.