Deterministic control of randomly-terminated processes. (English) Zbl 1292.49027
Summary: We consider both discrete and continuous “uncertain horizon” deterministic control processes, for which the termination time is a random variable. We examine the dynamic programming equations for the value function of such processes, explore their connections to infinite-horizon and optimal-stopping problems, and derive sufficient conditions for the applicability of non-iterative (label-setting) methods. In the continuous case, the resulting PDE has a free boundary, on which all characteristic curves originate. The causal properties of “uncertain horizon” problems can be exploited to design efficient numerical algorithms: we derive causal semi-Lagrangian and Eulerian discretizations for the isotropic randomly-terminated problems, and use them to build a modified version of the fast marching method. We illustrate our approach using numerical examples from optimal idle-time processing and expected response-time minimization.
MSC:
49L20 | Dynamic programming in optimal control and differential games |
49K45 | Optimality conditions for problems involving randomness |
49L25 | Viscosity solutions to Hamilton-Jacobi equations in optimal control and differential games |
93E20 | Optimal stochastic control |
35R35 | Free boundary problems for PDEs |
60G40 | Stopping times; optimal stopping problems; gambling theory |
65N22 | Numerical solution of discretized equations for boundary value problems involving PDEs |
90C39 | Dynamic programming |
05C85 | Graph algorithms (graph-theoretic aspects) |