Queueing theory

This article needs additional citations for verification. Please help improve this article by adding citations to reliable sources. Unsourced material may be challenged and removed. Find sources: "Queueing theory" – news · newspapers · books · scholar · JSTOR (January 2008) (Learn how and when to remove this message)

Queueing theory is the mathematical study of waiting lines, or queues.^[1] In queueing theory a model is constructed so that queue lengths and waiting times can be predicted.^[1] Queueing theory is generally considered a branch of operations research because the results are often used when making business decisions about the resources needed to provide service.

Queueing theory started with research by Agner Krarup Erlang when he created models to describe the Copenhagen telephone exchange.^[1] The ideas have since seen applications including telecommunications,^[2] traffic engineering, computing^[3] and the design of factories, shops, offices and hospitals.^[4][5]

The word queue comes, via French, from the Latin cauda, meaning tail. The spelling "queueing" over "queuing" is typically encountered in the academic research field. In fact, one of the flagship journals of the profession is named Queueing Systems.

Single queueing nodes are usually described using Kendall's notation in the form A/S/C where A describes the time between arrivals to the queue, S the size of jobs and C the number of servers at the node.^[6][7] Many theorems in queue theory can be proved by reducing queues to mathematical systems known as Markov chains, first described by Andrey Markov in his 1906 paper.^[8]

Agner Krarup Erlang, a Danish engineer who worked for the Copenhagen Telephone Exchange, published the first paper on what would now be called queueing theory in 1909.^[9][10][11] He modeled the number of telephone calls arriving at an exchange by a Poisson process and solved the M/D/1 queue in 1917 and M/D/k queueing model in 1920.^[12] In Kendall's notation

• M stands for Markov and means arrivals occur according to a Poisson process
• D stands for deterministic and means jobs arriving at the queue require a fixed amount of service
• k describes the number of servers at the queueing node (k = 1, 2,...). If there are more jobs at the node than there are servers then jobs will queue and wait for service.

The M/M/1 queue is a simple model where a single server serves jobs that arrive according to a Poisson process and have exponentially distributed service requirements. In an M/G/1 queue the G stands for general and indicates an arbitrary probability distribution. The M/G/1 model was solved by Felix Pollaczek in 1930, a solution later recast in probabilistic terms by Aleksandr Khinchin and now known as the Pollaczek–Khinchine formula.^[12] After World War II queueing theory became an area of research interest to mathematicians.^[12][13]

Work on queueing theory used in modern packet switching networks was performed in the early 1960s by Leonard Kleinrock. It was in this period that John Little gave a proof of the formula which know bears his name: Little's law^[14]. In 1961 John Kingman gave a formula for the mean waiting time in a G/G/1 queue: Kingman's formula^[15].

More recent results include Ramaswami’s formula^[16] for the M/G/1 queue.

The public switched telephone network (PSTN) is designed to accommodate the offered traffic intensity with only a small loss. The performance of loss systems is quantified by their grade of service, driven by the assumption that if sufficient capacity is not available, the call is refused and lost.^[17] Alternatively, overflow systems make use of alternative routes to divert calls via different paths — even these systems have a finite traffic carrying capacity.^[17]

However, the use of queueing in PSTNs allows the systems to queue their customers' requests until free resources become available. This means that if traffic intensity levels exceed available capacity, customer's calls are not lost; customers instead wait until they can be served.^[18] This method is used in queueing customers for the next available operator.

A queueing discipline determines the manner in which the exchange handles calls from customers.^[18] It defines the way they will be served, the order in which they are served, and the way in which resources are divided among the customers.^[18][19] Here are details of four queueing disciplines:

First in first out

This principle states that customers are served one at a time and that the customer that has been waiting the longest is served first.^[19]

Last in first out

This principle also serves customers one at a time, however the customer with the shortest waiting time will be served first.^[19] Also known as a stack.

Processor sharing

Service capacity is shared equally between customers.^[19]

Priority

Customers with high priority are served first.^[19]

Queueing is handled by control processes within exchanges, which can be modelled using state equations.^[18][19] Queueing systems use a particular form of state equations known as a Markov chain that models the system in each state.^[18] Incoming traffic to these systems is modelled via a Poisson distribution and is subject to Erlang’s queueing theory assumptions viz.^[17]

• Pure-chance traffic – Call arrivals and departures are random and independent events.^[17]
• Statistical equilibrium – Probabilities within the system do not change.^[17]
• Full availability – All incoming traffic can be routed to any other customer within the network.^[17]
• Congestion is cleared as soon as servers are free.^[17]

Classic queueing theory involves complex calculations to determine waiting time, service time, server utilization and other metrics that are used to measure queueing performance.^[18][19]

Networks of queues are systems a number of queues are connected by customer routing. When a customer is serviced at one node it can join another node and queue for service, or leave the network. For a network of m the state of the system can be described by an m–dimensional vector (x₁,x₂,...,x_m) where x_i represents the number of customers at each node. The first significant results in this area were Jackson networks^[20][21], for which an efficient product-form stationary distribution exists and the mean value analysis^[22] which allows average metrics such as throughput and sojourn times to be computed.^[23]

If the total number of customers in the network remains constant the network is called a closed network and has also been shown to have a product–form stationary distribution in the Gordon–Newell theorem^[24]. This result was extended to the BCMP network^[25] where a network with very general service time, regimes and customer routing is shown to also exhibit a product-form stationary distribution.

Networks of customers have also been investigated, Kelly networks where customers of different classes experience different priority levels at different service nodes.^[26]

Another type of network are G-networks first proposed by Erol Gelenbe in 1993^[27]: these networks do not assume exponential time distributions like the classic Jackson Network.

Mean field models consider the limiting behaviour of the empirical measure (proportion of queues in different states) as the number of queues (m above) goes to infinity. The impact of other queues on any given queue in the network is approximated by a differential equation. The deterministic model converges to the same stationary distribution as the original model.^[28]

Fluid models are continuous deterministic analogs of queueing networks obtained by taking the limit when the process is scaled in time and space, allowing heterogenous objects. This scaled trajectory converges to a deterministic equation which allows us stability of the system to be proven. It is known that a queueing network can be stable, but have an unstable fluid limit.^[29]

In a system with high occupancy rates (utilisation near 1) a heavy traffic approximation can be used to approximate the queueing length process by a reflected Brownian motion,^[30] Ornstein–Uhlenbeck process or more general diffusion process.^[31] The number of dimensions of the RBM is equal to the number of queueing nodes and the diffusion is restricted to the non-negative orthant.

Utilization is the proportion of the system's resources which is used by the traffic which arrives at it. It should be strictly less than one for the system to function well. It is usually represented by the symbol ${\displaystyle \rho }$. If ${\displaystyle \rho \geq 1}$ then the queue will continue to grow as time goes on. In the simplest case of an M/M/1 queue (Poisson arrivals and a single Poisson server) then it is given by the mean arrival rate over the mean service rate, that is,

${\displaystyle \rho ={\frac {\lambda }{\mu }}}$

where ${\displaystyle \lambda }$ is the mean arrival rate and ${\displaystyle \mu }$ is the mean service rate. More generally:

${\displaystyle \rho ={\frac {\lambda }{\mu \times c}}}$

where ${\displaystyle \lambda }$ is the mean arrival rate, ${\displaystyle \mu }$ is the mean service rate, and c is the number of servers, such as in an M/M/c queue.

In general, a lower utilization corresponds to less queueing for customers but means that the system is more idle, which may be considered inefficient.

A useful queueing model represents a real-life system with sufficient accuracy and is analytically tractable. A queueing model based on the Poisson process and its companion exponential probability distribution often meets these two requirements. A Poisson process models random events (such as a customer arrival, a request for action from a web server, or the completion of the actions requested of a web server) as emanating from a memoryless process. That is, the length of the time interval from the current time to the occurrence of the next event does not depend upon the time of occurrence of the last event. In the Poisson probability distribution, the observer records the number of events that occur in a time interval of fixed length. In the (negative) exponential probability distribution, the observer records the length of the time interval between consecutive events. In both, the underlying physical process is memoryless.

Models based on the Poisson process often respond to inputs from the environment in a manner that mimics the response of the system being modeled to those same inputs. The analytically tractable models that result yield both information about the system being modeled and the form of their solution. Even a queueing model based on the Poisson process that does a relatively poor job of mimicking detailed system performance can be useful. The fact that such models often give "worst-case" scenario evaluations appeals to system designers who prefer to include a safety factor in their designs. Also, the form of the solution of models based on the Poisson process often provides insight into the form of the solution to a queueing problem whose detailed behavior is poorly mimicked. As a result, queueing models are frequently modeled as Poisson processes through the use of the exponential distribution.

The assumptions of classical queueing theory may be too restrictive to be able to model real-world situations exactly. The complexity of production lines with product-specific characteristics cannot be handled with those models. Therefore specialized tools have been developed to simulate, analyze, visualize and optimize time dynamic queueing line behavior.

For example; the mathematical models often assume infinite numbers of customers, infinite queue capacity, or no bounds on inter-arrival or service times, when it is quite apparent that these bounds must exist in reality. Often, although the bounds do exist, they can be safely ignored because the differences between the real-world and theory is not statistically significant, as the probability that such boundary situations might occur is remote compared to the expected normal situation. Furthermore, several studies show the robustness of queueing models outside their assumptions. In other cases the theoretical solution may either prove intractable or insufficiently informative to be useful.

Alternative means of analysis have thus been devised in order to provide some insight into problems that do not fall under the scope of queueing theory, although they are often scenario-specific because they generally consist of computer simulations or analysis of experimental data. See network traffic simulation.

• Ehrenfest model
• Erlang unit
• Industrial engineering
• Markovian arrival processes
• Network simulation
• Queue area
• Queueing delay
• Random early detection
• Renewal theory
• Throughput
• Traffic generation model

• Gross, Donald (1998). Fundamentals of Queueing Theory. Wiley. ISBN 0-471-32812-X. {{cite book}}: Unknown parameter |coauthors= ignored (|author= suggested) (help)
• Deitel, Harvey M. (1984) [1982]. An introduction to operating systems (revisited first ed.). Addison-Wesley. p. 673. ISBN 0-201-14502-2. chap.15, pp. 380–412
• Lazowska, Edward D. (1984). Quantitative System Performance: Computer System Analysis Using Queueing Network Models. Prentice-Hall, Inc. ISBN 0-13-746975-6. {{cite book}}: Unknown parameter |coauthors= ignored (|author= suggested) (help)
• Zukerman, Moshe. Introduction to Queueing Theory and Stochastic Teletraffic Models (PDF).

• Queueing theory calculator
• Virtamo's Queueing Theory Course
• Myron Hlynka's Queueing Theory Page
• Queueing Theory Basics
• Java Modelling Tools - A GPL suite of queueing theory tools
• Queueing Package for GNU Octave
• A free online tool to solve some classical queueing systems