Abstract
We consider Markov Decision Processes under light traffic conditions. We develop an algorithm to obtain asymptotically optimal policies for both the total discounted and the average cost criterion. This gives a general framework for several light traffic results in the literature. We illustrate the method by deriving the asymptotically optimal control of a simple ATM network.
Similar content being viewed by others
References
Blanc JPC (1993) Performance analysis and optimization with the power-series algorithm. In: Donatiello L, Nelson R (Eds.) Performance evaluation of computer and communication systems. Lecture notes in computer sciences 729, Springer-Verlag: 53–80
Derman C (1970) Finite state Markov decision processes. Academic Press New York
Katehakis MM, Derman C (1989) On the maintenance of system composed of highly reliable components. Management Science 35(5):551–560
Katehakis MN, Levine A (1985) A dynamic routing problem — numerical procedures for light traffic conditions. Applied Mathematics and Computation 17:267–276
Katehakis MN, Levine (1986) Allocation of distinguishable servers. Computers and Operations Research 13:85–93
Koole G (1994) On the power series algorithm. In: Boxma O, Koole G (Eds.) Performance Evaluation of Parallel and Distributed Systems — Solution Methods. CWI Tract 105, CWI, Amsterdam. To appear in ORSA Journal of Computing: 139–155
Koole G, Liu Z (1995) Nonconservative service for minimising cell loss in ATM networks. Proceedings of WATM '95, Paris: 123–130
Koole G, Vrijenhoek M (1995) Scheduling a repairman in a finite source system. Technical Report TW-95-03, Leiden University To appear in ZOR 1996
Levine A, Finkel D (1990) Load balancing in a multiserver queueing system. Computers and Operations Research 17(1):17–24
Passchier O (1996) The theory of Markov games and queueing control. PhD. Thesis, Leiden University
Smith DR (1978) Optimal repairman allocation — asymptotical results. Management Science 24:665–674
Author information
Authors and Affiliations
Rights and permissions
About this article
Cite this article
Koole, G., Passchier, O. Optimal control in light traffic Markov decision processes. Mathematical Methods of Operations Research 45, 63–79 (1997). https://doi.org/10.1007/BF01194248
Received:
Revised:
Issue Date:
DOI: https://doi.org/10.1007/BF01194248