Summary: The neighbor Q-learning based consensus control algorithm is developed for discrete-time multi-agent systems in this article. To realize the proposed algorithm, a new actor-critic architecture is employed for each agent. The critic network of each agent approximates its Q-function while the actor network produces control signal by minimizing the Q-function. Considering the distribution metrics of the systems, the neighbors’ Q-functions of each agent are applied to the update procedure of the critic network to stabilize the learning process and avoid the overestimation problem. The convergence properties and stability analysis for the proposed algorithm are provided. Different discount factors corresponding to various topology structures are discussed in the convergence analysis section. The accurate system model is nonessential for the algorithm which is too intricate to build up for practical systems. Finally, three simulation examples including different discount factors are conducted to demonstrate the effectiveness of the consensus control algorithm.
{© 2022 John Wiley & Sons Ltd.}