Incentive Stackelberg games for stochastic systems. (English) Zbl 1448.91069
Yeung, David (ed.) et al., Frontiers in games and dynamic games. Theory, applications, and numerical methods. Selected papers based on the presentations at the ISDG-China chapter conference on dynamic games and game theoretic analysis, Ningbo, China, August 3–5, 2017. Cham: Birkhäuser. Ann. Int. Soc. Dyn. Games 16, 41-118 (2020).
Summary: Dynamic games with hierarchical structure have been identified as key components of modern control systems that enable the integration of renewable cooperative and/or noncooperative control such as distributed multi-agent systems. Although the incentive Stackelberg strategy has been admitted as the hierarchical strategy that induces the behavior of the decision maker as that of the follower, the followers optimize their costs under incentives without a specific information. Therefore, leaders succeed in using the required strategy to induce the behavior of their followers. This concept is considered very useful and reliable in some practical cases. In this survey, incentive Stackelberg games for deterministic and stochastic linear systems with external disturbance are addressed. The induced features of the hierarchical strategy in the considered models, including stochastic systems governed by Itô stochastic differential equation, Markov jump linear systems, and linear parameter varying (LPV) systems, are explained in detail. Furthermore, basic concepts based on the \(H_2/H_\infty\) control setting for the incentive Stackelberg games are reviewed. Next, it is shown that the required set of strategies can be designed by solving higher-order cross-coupled algebraic Riccati-type equations. Finally, as a partial roadmap for the development of the underdeveloped pieces, some open problems are introduced.
For the entire collection see [Zbl 1446.91014].
For the entire collection see [Zbl 1446.91014].
MSC:
| 91A65 | Hierarchical games (including Stackelberg games) |
| 91A25 | Dynamic games |
| 93E03 | Stochastic systems in control theory (general) |
| 93A16 | Multi-agent systems |
Keywords:
incentive Stackelberg games; Nash games; Pareto optimal strategy; \(H_\infty\) constraint; stochastic systems; Markov jump systems; linear parameter varying systemsReferences:
| [1] | T. Basar and G. J. Olsder, Dynamic Noncooperative Game Theory, Philadelphia: SIAM Series in Classics in Applied Mathematics, 1999. · Zbl 0946.91001 |
| [2] | V. R. Saksena and J. B. Cruz, Jr, Optimal and near-optimal incentive strategies in the hierarchical control of Markov chains, Automatica, vol. 21, no. 2, pp. 181-191, 1985. · Zbl 0564.90105 · doi:10.1016/0005-1098(85)90112-8 |
| [3] | K. B. Kim, A. Tang and S. H. Low, “A stabilizing AQM based on virtual queue dynamics in supporting TCP with arbitrary delays,” in Proc. 42nd IEEE Conf. Decision and Control, Maui, HI, December 2003, pp. 3665-3670. |
| [4] | R. Kicsiny, Solution for a class of closed-loop leader-follower games with convexity conditions on the payoffs, Annals of Operations Research, vol. 253, no. 1, pp. 405-429, 2017. · Zbl 1406.91075 · doi:10.1007/s10479-016-2327-9 |
| [5] | M. Yu and S. H. Hong, Supply-demand balancing for power management in smart grid: A Stackelberg game approach, Applied Energy, vol. 164, pp. 702-710, 2016. · doi:10.1016/j.apenergy.2015.12.039 |
| [6] | C. I. Chen and J. B. Cruz, Jr., Stackelberg solution for two-person games with biased information patterns, IEEE Trans. Automatic Control, vol. 17, no. 6, pp. 791-798, Dec. 1972. · Zbl 0262.90095 · doi:10.1109/TAC.1972.1100179 |
| [7] | J. V. Medanic, Closed-loop Stackelberg strategies in linear-quadratic problems, IEEE Trans. Automatic Control, vol. 23, no. 4, pp. 632-637, Aug. 1978. · Zbl 0381.49006 · doi:10.1109/TAC.1978.1101788 |
| [8] | M. Jungers, E. Trelat and H. Abou-Kandil, Min-max and min-min Stackelberg strategies with closed-loop information structure, J. Dynamical and Control Systems, vol. 17, no. 3, pp. 387-425, 2011. · Zbl 1228.91019 · doi:10.1007/s10883-011-9123-2 |
| [9] | A. Bensoussan, S. Chen and S. P. Sethi, The maximum principle for global solutions of stochastic Stackelberg differential games, SIAM Control and Optimization, vol. 53, no. 4, pp. 1956-1981, 2015. · Zbl 1320.91042 · doi:10.1137/140958906 |
| [10] | J. Xu and H. Zhang, Sufficient and necessary open-loop Stackelberg strategy for two-player game with time delay, IEEE Trans. Cybernetics, vol. 46, no. 2, pp. 438-449, Feb. 2016. · doi:10.1109/TCYB.2015.2403262 |
| [11] | H. Mukaidani and H. Xu, Stackelberg strategies for stochastic systems with multiple followers, Automatica, vol. 53, pp. 53-59, 2015. · Zbl 1371.93217 · doi:10.1016/j.automatica.2014.12.021 |
| [12] | H. Mukaidani and H. Xu, Infinite-horizon linear-quadratic Stackelberg games for discrete-time stochastic systems, Automatica, vol. 76, no. 5, pp. 301-308, 2017. · Zbl 1352.93106 · doi:10.1016/j.automatica.2016.10.016 |
| [13] | Y. C. Ho, Peter B. Luh and G. J. Olsder, A control-theoretic view on incentives, Automatica, vol. 18, no. 2, pp. 167-179, 1982. · Zbl 0477.90003 |
| [14] | T. Basar and H. Selbuz, Closed-loop Stackelberg strategies with applications in the optimal control of multilevel systems, IEEE Trans. Automatic Control, vol. 24, no. 2, pp. 166-178, April 1979. · Zbl 0405.49020 · doi:10.1109/TAC.1979.1101999 |
| [15] | B. Tolwinski, Closed-loop Stackelberg solution to multi-stage linear quadratic game, J. Optimization Theory and Applications, vol. 34, no. 3, pp. 485-501, 1981. · Zbl 0431.90097 · doi:10.1007/BF00935889 |
| [16] | Y. P. Zheng and T. Basar, Existence and derivations of optimal affine incentive schemes for Stackelberg games with partial information: A geometric approach, Int. J. Control, vol. 35, no. 6, pp. 997-1011, 1982. · Zbl 0484.90099 · doi:10.1080/00207178208922667 |
| [17] | Y. P. Zheng, T. Basar and J. B. Cruz Jr., Stackelberg strategies and incentives in multiperson deterministic decision problems, IEEE Trans. Systems, Man, and Cybernetics, vol. 14, no. 1, pp. 10-24, Jan. 1984. · Zbl 0541.90107 · doi:10.1109/TSMC.1984.6313265 |
| [18] | T. Basar and G. J. Olsder, Team-optimal closed-loop Stackelberg strategies in hierarchical control problems, Automatica, vol. 16, no. 3, pp. 409-414, 1980. · Zbl 0444.93004 · doi:10.1016/0005-1098(80)90026-6 |
| [19] | P. B. Luh, S. C. Chang and T.S. Chang, Solutions and properties of multi-stage Stackelberg games, Automatica, vol. 20, no. 2, pp. 251-256, 1984. · Zbl 0529.90105 · doi:10.1016/0005-1098(84)90034-7 |
| [20] | T.S. Chang and P. B. Luh, Derivation of necessary and sufficient conditions for single-state Stackelberg games via the inducible region concept, IEEE Trans. Automatic Control, vol. 29, no. 1, pp. 63-66, 1984. · Zbl 0536.90097 · doi:10.1109/TAC.1984.1103381 |
| [21] | P. B. Luh, T. S. Chang and T. Ning, Three-level Stackelberg decision problems, IEEE Trans. Automatic Control, vol. 29, no. 3, pp. 280-282, 1984. · Zbl 0529.90104 · doi:10.1109/TAC.1984.1103503 |
| [22] | K. Mizukami and H. Wu, Two-level incentive Stackelberg strategies in LQ differential games with two noncooperative leaders and one follower, Trans. SICE, vol. 23, no. 6, pp. 625-632, 1987. · doi:10.9746/sicetr1965.23.625 |
| [23] | K. Mizukami and H. Wu, Incentive Stackelberg strategies in linear quadratic differential games with two noncooperative followers, Modelling And Methodology In Social And Economic Systems System Modelling and Optimization Volume 113 of the series Lecture Notes in Control and Information Sciences, pp. 436-445, Berlin Heidelberg: Springer, 1988. · Zbl 0655.90107 |
| [24] | T. Ishida and E. Shimemura, Three-level incentive strategies in differential games, Int. J. Control, vol. 38, no. 6, 1983, pp. 1135-1148. · Zbl 0531.90106 · doi:10.1080/00207178308933135 |
| [25] | T. Basar, Equilibrium strategies in dynamic games with multilevel of hierarchy, Automatica, vol. 17, no. 5, pp. 749-754, 1981. · Zbl 0474.93005 · doi:10.1016/0005-1098(81)90022-4 |
| [26] | M. Li, J. B. Cruz Jr., M. A. Simaan, An approach to discrete-time incentive feedback Stackelberg games, IEEE Trans. Systems, Man, and Cybernetics, vol. 32, no. 4, pp. 10-24, July 2002. |
| [27] | H. Mukaidani, Infinite-horizon team-optimal incentive Stackelberg games for linear stochastic systems, IEICE Trans. Fundamentals of Electronics, Communications and Computer Sciences, vol. E99-A, no. 9, pp. 1721-1725, 2016. · doi:10.1587/transfun.E99.A.1721 |
| [28] | M. Ahmed and H. Mukaidani, “H_∞-constrained incentive Stackelberg game for discrete-time systems with multiple non-cooperative followers,” in Proc. 6th IFAC Workshop on Distributed Estimation and Control in Networked Systems, Tokyo, Japan, September 2016, IFAC-PapersOnLine, 49-22, pp. 262-267. |
| [29] | H. Mukaidani, M. Ahmed, T. Shima and H. Xu, “H_∞ constraint incentive Stackelberg game for discrete-time stochastic systems,” in Proc. American Control Conf., Seattle, WA, May 2017, pp. 5257-5262. |
| [30] | M. Ahmed, H. Mukaidani and T. Shima, “Infinite-horizon multi-leader-follower incentive Stackelberg games for linear stochastic systems with H_∞ constraint,” in SICE Annual Conf., Kanazawa, Japan 2017, pp. 1202-1207. |
| [31] | M. Ahmed, H. Mukaidani and T. Shima, H_∞-constrained incentive Stackelberg games for discrete-time stochastic systems with multiple followers, IET Control Theory and Applications, vol. 11, no. 15, pp. 2475-2485, 2017. · doi:10.1049/iet-cta.2017.0105 |
| [32] | H. Mukaidani and T. Shima, M. Unno, H. Xu and V. Dragan, “Team-optimal incentive Stackelberg strategies for Markov jump linear stochastic systems with H_∞ constraint,” in Proc. 20th IFAC World Congress, Toulouse, France, July 2017, IFAC-PapersOnLine, 50-1, pp. 3780-3785. |
| [33] | H. Mukaidani, H. Xu and V. Dragan, A stochastic multiple-leader follower incentive Stackelberg strategy for Markov jump linear systems, IEEE Control Systems Letters, vol. 1, no. 2, pp. 250-255, 2017. (See also 56th IEEE Conf. Decision and Control, pp. 3688-3693, Melbourne, Australia, December 2017.) |
| [34] | H. Mukaidani, H. Xu and V. Dragan, Static output-feedback incentive Stackelberg game for discrete-time Markov jump linear stochastic systems with external disturbance, IEEE Control Systems Letters, vol. 2, no. 4, pp. 701-706, 2018. (See also 57th IEEE Conf. Decision and Control, Miami, FL, December 2018.) |
| [35] | H. Mukaidani and H. Xu, Incentive Stackelberg games for stochastic linear systems with H_∞ constraint, IEEE Trans. Cybernetics, vol. 49, no. 4, pp. 1463-1474, 2019. · doi:10.1109/TCYB.2018.2804920 |
| [36] | K. Kawakami, H. Mukaidani, H. Xu and Y. Tanaka, “Incentive Stackelberg-Nash strategy with disturbance attenuation for stochastic LPV systems,” in Proc. the 2018 IEEE Int. Conf. Systems, Man, and Cybernetics, Miyazaki, Japan, October 2018, pp. 3940-3945. |
| [37] | H. Mukaidani, H. Xu, T. Shima and M. Ahmed, “Multi-leader-follower incentive Stackelberg game for infinite-horizon Markov jump linear stochastic systems with H_∞ constraint,” in Proc. the 2018 IEEE Int. Conf. Systems, Man, and Cybernetics, Miyazaki, Japan, October 2018, pp. 3946-3953. |
| [38] | H. Mukaidani and Hua Xu, “Robust incentive Stackelberg games for stochastic LPV systems,” in Proc. 57th IEEE Conf. Decision and Control, Miami, FL, December 2018, pp. 1059-1064. |
| [39] | B. S. Chen and W. Zhang. Stochastic H_2∕H_∞ control with state-dependent noise, IEEE Trans. Automatic Control, vol. 49, no. 1, pp. 45-57, Jan. 2004. · Zbl 1365.93539 · doi:10.1109/TAC.2003.821400 |
| [40] | W. Zhang and B. -S. Chen, On stabilizability and exact observability of stochastic systems with their applications, Automatica, vol. 40, no. 1, pp. 87-94, 2004. · Zbl 1043.93009 · doi:10.1016/j.automatica.2003.07.002 |
| [41] | J. C. Engwerda, LQ Dynamic Optimization and Differential Games, Chichester: John Wiley and Sons, 2005. |
| [42] | H. Mukaidani, Soft-constrained stochastic Nash games for weakly coupled large-scale systems, Automatica, vol. 45, no. 5, pp. 1272-1279, 2009. · Zbl 1162.93310 · doi:10.1016/j.automatica.2008.12.020 |
| [43] | Z. Gajic and M. Qureshi, Lyapunov Matrix Equation in System Stability and Control, San Diego, Academic Press, Mathematics in Science and Engineering Series, 1995. · Zbl 1153.93300 |
| [44] | M. Sagara, H. Mukaidani and T. Yamamoto, Numerical solution of stochastic Nash games with state-dependent noise for weakly coupled large-scale systems, Applied Mathematics and Computation, vol. 197, no. 2, pp 844-857, 2008. · Zbl 1136.91324 · doi:10.1016/j.amc.2007.08.019 |
| [45] | V. Dragan and T. Morozan, The linear quadratic optimization problems for a class of linear stochastic systems with multiplicative white noise and Markovian jumping, IEEE Trans. Automatic Control, vol. 49, no. 5, pp. 665-675, 2004. · Zbl 1365.93541 · doi:10.1109/TAC.2004.826718 |
| [46] | Y. Huang, W. Zhang and G. Feng, Infinite-horizon H_2∕H_∞ control for stochastic systems with Markovian jumps, Automatica, vol. 44, no. 3, pp. 857-863, 2008. · Zbl 1283.93253 · doi:10.1016/j.automatica.2007.07.001 |
| [47] | X. Li, X. Y. Zhou and M. A. Rami, Indefinite stochastic linear quadratic control with Markovian jumps in infinite time horizon, J. Global Optimization, vol. 27, no. 2-3, pp. 149-175, 2003. · Zbl 1031.93155 · doi:10.1023/A:1024887007165 |
| [48] | H. Mukaidani, “Gain-scheduled H_∞ constraint Pareto optimal strategy for stochastic LPV systems with multiple decision makers,” in American Control Conf., Seattle, WA, May 2017, pp. 1097-1102. |
| [49] | W. Zhang and G. Feng, Nonlinear stochastic H_2∕H_∞ control with (x, u, v)-dependent noise: Infinite-horizon case, IEEE Trans. Automatic Control, vol. 53, no. 5, pp 1323-1328, 2008. · Zbl 1367.93192 |
| [50] | M. A. Rami and X. Y. Zhou, Linear matrix inequalities, Riccati equations, and indefinite stochastic linear quadratic controls, IEEE Trans. Automatic Control, vol. 45, no. 6, pp 1131-1143, 2000. · Zbl 0981.93080 · doi:10.1109/9.863597 |
| [51] | C. · doi:10.1115/1.4031059 |
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. In some cases that data have been complemented/enhanced by data from zbMATH Open. This attempts to reflect the references listed in the original paper as accurately as possible without claiming completeness or a perfect matching.
