Summary: In dial-a-ride problems, a fleet of \(n\) vehicles is routed to transport people between pick-up and delivery locations. We consider an elementary version of the problem where trip requests arrive in time and require an immediate vehicle assignment (which triggers an appropriate route update of the selected vehicle). In this context, a relatively general objective can be stated as a weighted sum of the system’s effort and the customers’ inconvenience. However, optimizing almost any objective in this immensely complex stochastic system is prohibitively difficult. Thus the earlier work has largely resorted to heuristic cost functions that arise, e.g., from the corresponding static systems. By using the framework of Markov decision processes and the classical M/M/1 queue as a highly abstract model for a single vehicle, we explain why certain intuitive cost functions indeed give satisfactory results in the dynamic system, and also give an explicit interpretation of different components appearing in a general cost function. The resulting family of heuristic control policies is demonstrated to offer a desired type of performance thus justifying the assumed analogy between a multi-queue and dial-a-ride systems.