Abstract
This paper is concerned with stochastic linear quadratic (LQ, for short) optimal control problems in an infinite horizon with constant coefficients. It is proved that the non-emptiness of the admissible control set for all initial state is equivaleznt to the \(L^{2}\)-stabilizability of the control system, which in turn is equivalent to the existence of a positive solution to an algebraic Riccati equation (ARE, for short). Different from the finite horizon case, it is shown that both the open-loop and closed-loop solvabilities of the LQ problem are equivalent to the existence of a static stabilizing solution to the associated generalized ARE. Moreover, any open-loop optimal control admits a closed-loop representation. Finally, the one-dimensional case is worked out completely to illustrate the developed theory.
Similar content being viewed by others
References
Ait Rami, M., Moore, J.B., Zhou, X.Y.: Indefinite stochastic linear quadratic control and generalized differential Riccati equation. SIAM J. Control Optim. 40, 1296–1311 (2001)
Ait Rami, M., Zhou, X.Y.: Linear matrix inequalities, Riccati equations, and indefinite stochastic linear quadratic controls. IEEE Trans. Automat. Control 45, 1131–1143 (2000)
Ait Rami, M., Zhou, X.Y., Moore, J.B.: Well-posedness and attainability of indefnite stochastic linear quadratic control in infnite time horizon. Syst. Control Lett. 41, 123–133 (2000)
Anderson, B.D.O., Moore, J.B.: Optimal Control: Linear Quadratic Methods. Prentice Hall, Englewood Cliffs (1989)
Bensoussan, A.: Lectures on Stochastic Control, Part I, in Nonlinear Filtering and Stochastic Control. Lecture Notes in Mathematics, vol. 972. Springer, Berlin (1982)
Chen, S., Li, X., Zhou, X.Y.: Stochastic linear quadratic regulators with indefinite control weight costs. SIAM J. Control Optim. 36, 1685–1702 (1998)
Chen, S., Yong, J.: Stochastic linear quadratic optimal control problems. Appl. Math. Optim. 43, 21–45 (2001)
Chen, S., Zhou, X.Y.: Stochastic linear quadratic regulators with indefinite control weight costs. II. SIAM J. Control Optim. 39, 1065–1081 (2000)
Davis, M.H.A.: Linear Estimation and Stochastic Control. Chapman and Hall, London (1977)
Hu, Y., Zhou, X.Y.: Indefinite stochastic Riccati equations. SIAM J. Control Optim. 42, 123–137 (2003)
Huang, J., Li, X., Yong, J.: A linear-quadratic optimal control problem for mean-field stochastic differential equations in infinite horizon. Math. Control & Relat. Fields 5, 97–139 (2015)
Kalman, R.E.: Contributions to the theory of optimal control. Bol. Soc., Mat. Mex. 5, 102–119 (1960)
Li, X., Sun, J., Yong, J.: Mean-field stochastic linear quadratic optimal control problems: closed-loop solvability. Probab. Uncertain. Quant. Risk 1, 2 (2016). doi:10.1186/s41546-016-0002-3
Lim, A.E.B., Zhou, X.Y.: Stochastic optimal LQR control with integral quadratic constraints and indefinite control weights. IEEE Trans. Automat. Control 44, 359–369 (1999)
Penrose, R.: A generalized inverse of matrices. Proc. Camb. Philos. Soc. 52, 17–19 (1955)
Qian, Z., Zhou, X.Y.: Existence of solutions to a class of indefinite stochastic Riccati equations. SIAM J. Control Optim. 51, 221–229 (2013)
Sun, J.: Mean-field stochastic linear quadratic optimal control problems: Open-loop solvabilities. ESAIM: COCV. doi:10.1051/cocv/2016023 (2016)
Sun, J., Li, X., Yong, J.: Open-loop and closed-loop solvabilities for stochastic linear quadratic optimal control problems. SIAM J. Control Optim. 54, 2274–2308 (2016)
Sun, J., Yong, J.: Linear quadratic stochastic differential games: open-loop and closed-loop saddle points. SIAM J. Control Optim. 52, 4082–4121 (2014)
Sun, J., Yong, J., Zhang, S.: Linear quadratic stochastic two-person zero-sum differential games in an infinite horizon. ESAIM: COCV 22, 743–769 (2016)
Wonham, W.M.: On a matrix Riccati equation of stochastic control. SIAM J. Control 6, 681–697 (1968)
Wu, H., Zhou, X.Y.: Stochastic frequency characteristics. SIAM J. Control Optim. 40, 557–576 (2001)
Yao, D.D., Zhang, S., Zhou, X.Y.: Stochastic linear-quadratic control via semidefinite programming. SIAM J. Control Optim. 40, 801–823 (2001)
Yong, J.: Linear-quadratic optimal control problems for mean-field stochastic differential equations. SIAM J. Control Optim. 51, 2809–2838 (2013)
Yong, J., Zhou, X.Y.: Stochastic Controls: Hamiltonian Systems and HJB Equations. Springer, New York (1999)
You, Y.: Optimal control for linear system with quadratic indefinite criterion on Hilbert spaces. Chinese Ann. Math. Ser. B 4, 21–32 (1983)
Acknowledgements
The authors would like to thank the anonymous referees for their suggestive comments, which lead to an improvement of the paper. Jiongmin Yong was partially supported by NSF Grant DMS-1406776.
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Sun, J., Yong, J. Stochastic Linear Quadratic Optimal Control Problems in Infinite Horizon. Appl Math Optim 78, 145–183 (2018). https://doi.org/10.1007/s00245-017-9402-8
Published:
Issue Date:
DOI: https://doi.org/10.1007/s00245-017-9402-8
Keywords
- Stochastic linear quadratic optimal control
- Stabilizability
- Open-loop solvability
- Closed-loop solvability
- Algebraic Riccati equation
- Static stabilizing solution
- Closed-loop representation