Summary: We propose a framework for solving complex decision problems based on a partition in simpler problems that can be solved independently, and then combined to obtain an optimal, global solution. Each aspect of the problem is represented as an MDP and solved independently. At any state of the problem, each MDP sends its value for each possible action (its \(Q\) value) and an arbiter selects the action with the greatest combined value. In contrast to previous approaches for hierarchical MDPs, in our approach all the MDPs work in parallel, so we obtain a reactive system based on a decision theoretical framework. We present an algorithm for solving parallel MDPs and prove it obtains the global optimum, assuming an additive value. We present experimental results in two cases: (i) a simulated robot navigation problem, (ii) a real robot in a message delivery task.
For the entire collection see [Zbl 1108.90003].