From the authors’ preface: “The purpose of this book is to collect together newest results of the theory of Markov decision processes related to queueing models and to demonstrate their applications to main types of control in queueing systems. These types are: control of arrivals; control of service mechanism; control of service discipline.”
The authors concentrate on monotonicity, threshold or histeretic character, priority and other structural properties of Markov optimal strategies. These properties allow to restrict the range of search of optimal strategy and sometimes to construct it in an explicit form. The authors hope “that this book will turn out useful for applied mathematicians interested in queueing systems, specialists in system analysis ... for graduate and postgraduate students ... we restrict ourselves to the case of denumerable state space ... the reader is assumed to be familiar with standard courses of analysis, probability theory and queueing theory.” Each chapter includes a set of exercises which are the essential part of the book.
The material is organized in eight chapters. Chapters 1-5 may be unified by the general idea “Methods” and Chapters 6-8 by “Models”. In more detail these chapters are the following: 1. Semi-regenerative decision models; 2. Dynamic programming; 3. Average cost criterion; 4. Continuously controlled Markov jump processes; 5. Structured optimization problems for decision processes; 6. Control of arrivals; 7. Control of service mechanism; 8. Optimal control in models with several classes of customers.