Summary: We treat the average cost per unit time problem for controlled and uncontrolled open queueing networks in heavy traffic. The usual heavy traffic theorems prove that a suitably scaled and normalized sequence of queue length processes converges weakly in the Skorokhod topology to a certain reflected diffusion, as the traffic intensity goes to unity. The Skorokhod topology essentially discounts the distant future, and the usual weak convergence methods are not well suited for dealing with the average cost problem over an infinite time interval. We provide a particularly strong approach to this problem via a “functional occupation measure” method. A fairly general cost functional is used. For the uncontrolled problem, it is shown that the average pathwise (i.e., no mathematical expectation is used) cost per unit time converges in probability to an ergodic cost for the limit reflected diffusion, no matter how the time goes to infinity or the traffic intensity goes to unity.
The methods which are introduced are new and are quite powerful for getting limit and approximation results for many types of controlled or uncontrolled sequences of systems over arbitrarily large time intervals. The methods involve adaptations of the martingale problem methods to sequences of occupation measures (defined by the sample paths) over the path space. They are used to characterize the stationary processes (controlled or not) which are defined by the values of the weak limits of the functional occupation measures. The ergodic costs for these processes are just the pathwise limits of the costs for the sequence of physical problems. For the controlled problem, it is shown that nearly optimal controls for the “mean” ergodic cost for the limit model are also nearly optimal for the pathwise average cost for the physical problem. The method is applicable to generate representation and control problems on the infinite time interval for sequences of processes.