A partial history of the early development of continuous-time nonlinear stochastic systems theory. (English) Zbl 1364.93866
Summary: This article is a survey of the early development of selected areas in nonlinear continuous-time stochastic control. Key developments in optimal control and the dynamic programming principle, existence of optimal controls under complete and partial observations, nonlinear filtering, stochastic stability, the stochastic maximum principle and ergodic control are discussed. Issues concerning wide bandwidth noise for stability, modeling, filtering and ergodic control are dealt with. The focus is on the earlier work, but many important topics are omitted for lack of space.
MSC:
| 93E20 | Optimal stochastic control |
| 49L20 | Dynamic programming in optimal control and differential games |
| 49J55 | Existence of optimal solutions to problems involving randomness |
| 49K45 | Optimality conditions for problems involving randomness |
| 93E11 | Filtering in stochastic control theory |
| 93E10 | Estimation and detection in stochastic control theory |
| 93E15 | Stochastic stability in control theory |
Keywords:
stochastic optimal control; viscosity solutions; dynamic programming principle; Nisio semigroup; existence of optimal controls; existence under partial observations; compactness methods; wide bandwidth noise; approximations to systems; nonlinear filtering; stochastic stability; stochastic maximum principle; ergodic control; stochastic stability; filtering and control with wide bandwidth noiseReferences:
| [1] | Allinger, D. F.; Mitter, S. K., New results on the innovations problem for nonlinear filtering, Stochastics, 4, 339-348 (1980-81) · Zbl 0453.60046 |
| [2] | Arasaratnam, I.; Haykin, S.; Elliott, R. J., Discrete-time nonlinear filtering algorithms using Gauss-Hermite quadrature, IEEE Transactions, 95, 953-977 (2007) |
| [3] | Arnold, L.; Kliemannn, W.; Oeljeklaus, E., Liapunov exponents of linear stochastic systems, (Arnold, L.; Wihstutz, V., Liapunov exponents. Liapunov exponents, Lecture notes in math., Vol. 1186 (1986), Springer-Verlag: Springer-Verlag Berlin and New York) |
| [4] | Aström, K. J., Introduction to stochastic control theory (1970), Academic Press: Academic Press New York · Zbl 0226.93027 |
| [5] | Atar, R.; Budhiraja, A.; Dupuis, P., On positive recurrence of constrained diffusion processes, The Annals of Probability, 29, 979-1000 (2001) · Zbl 1018.60081 |
| [6] | Atar, R.; Zeitouni, O., Exponential stability for nonlinear filtering, Annales de l’Institut Henri Poincaré, 33, 697-725 (1997) · Zbl 0888.93057 |
| [7] | Athans, M.; Falb, P., Optimal control (1966), McGraw Hill: McGraw Hill New York · Zbl 0186.22501 |
| [8] | Balakrishnan, A. V., Stochastic control: a function space approach, SIAM Journal on Control, 10, 285-297 (1972) · Zbl 0242.93050 |
| [9] | Bar Shalom, Y.; Li, X.-R., Estimation and tracking: principles, techniques and software (1993), Artech House: Artech House Norwood MA · Zbl 0853.62076 |
| [10] | Bardi, M.; Capuzzo-Dolcetta, I., Optimal control and viscosity solutions of Hamilton-Jacobi-Bellman equations (1997), Birkhäuser: Birkhäuser Boston · Zbl 0890.49011 |
| [11] | Basar, T.; Bernhard, P., \(H_\infty \)-optimal control and related minimax problems (1991), Birkhäuser: Birkhäuser Boston · Zbl 0751.93020 |
| [12] | Bellman, R., On the theory of dynamic programming, Proceedings of the National Academy of Sciences, 38, 716-719 (1952) · Zbl 0047.13802 |
| [13] | Bellman, R., Dynamic programming and a new formalism in the calculus of variations, Proceedings of the National Academy of Sciences, 40, 231-235 (1954) · Zbl 0058.36303 |
| [14] | Bellman, R., Dynamic programming (1957), Princeton Univ. Press: Princeton Univ. Press Princeton NJ · Zbl 0077.13605 |
| [15] | Bellman, R., Adaptive control processes: a guided tour (1961), Princeton Univ. Press: Princeton Univ. Press Princeton, NJ · Zbl 0103.12901 |
| [16] | Benes˘, V. E., Existence of finite invariant measures for Markov processes, Proceedings of the Americal Mathematical Society, 18, 1058-1061 (1967) · Zbl 0241.60059 |
| [17] | Benes˘, V. E., Extension of Clark’s innovation equivalence theorem, (Stochastic systems: modeling, identification and optimization I. Stochastic systems: modeling, identification and optimization I, Math. prog. studies, Vol. 5 (1976), Springer-Verlag: Springer-Verlag Berlin and New York), 2-7 · Zbl 0357.00019 |
| [18] | Benes˘, V. E., Exact finite-dimensional filters for certain diffusions with nonlinear drift, Stochastics, 5, 65-82 (1981) · Zbl 0458.60030 |
| [19] | Benes˘, V. E., Existence of optimal stochastic control laws, SIAM Journal on Control and Optimization, 9, 446-475 (1971) · Zbl 0219.93029 |
| [20] | Bensoussan, A., Lectures on stochastic control, (Mitter, S.; Moro, A., Nonlinear filtering and stochastic control. Nonlinear filtering and stochastic control, Lecture note in math., Vol. 972 (1982), Springer-Verlag: Springer-Verlag New York and Berlin), 1-62 · Zbl 0505.93078 |
| [21] | Bensoussan, A., Stochastic control by functional analysis methods (1982), Elsevier: Elsevier Amsterdam · Zbl 0474.93002 |
| [22] | Bensoussan, A., Maximum principle and dynamic programming approaches of the optimal control of partially observed diffusions, Stochastics, 9, 169-222 (1983) · Zbl 0516.60072 |
| [23] | Bensoussan, A., Perturbation methods in optimal control (1988), Wiley: Wiley New York · Zbl 0648.49001 |
| [24] | Bensoussan, A., (On the integrated formulation of Zakai and Kushner equations. On the integrated formulation of Zakai and Kushner equations, Lecture notes in math., Vol. 1390 (1989), Springer-Verlag: Springer-Verlag Berlin and New York), 13-23 · Zbl 0677.60066 |
| [25] | Bensoussan, A., Stochastic control of partially observed diffusions (1992), Cambridge Univ. Press: Cambridge Univ. Press Cambridge · Zbl 0502.93075 |
| [26] | Bensoussan, A.; Frehse, J., On Bellman equations of ergodic control in \(R^n\), Journal für die Reine und Angewandte Mathematik, 429, 125-160 (1992) · Zbl 0779.35038 |
| [27] | Bensoussan, A.; Frehse, J., Ergodic control Bellman equation with Neumann boundary conditions, (Stochastic theory and control: lecture notes in cont. and inform. sci., Vol. 280 (2002), Springer-Verlag: Springer-Verlag Berlin and New York), 59-71 · Zbl 1050.93075 |
| [28] | Bensoussan, A.; Lions, J., Applications des inéquations variationnelles en contrôl stochastique (1978), Dunod: Dunod Paris · Zbl 0411.49002 |
| [29] | Bensoussan, A.; Lions, J., Applications of variational inequalities in stochastic control (1982), North Holland: North Holland Amsterdam · Zbl 0478.49002 |
| [30] | Benveniste, A.; Metivier, M.; Priouret, P., Adaptive algorithms and stochastic approximation (1990), Springer-Verlag: Springer-Verlag Berlin and New York · Zbl 0752.93073 |
| [31] | Berkovitz, L., (Optimal control theory. Optimal control theory, Applied mathematical sciences., Vol. 12 (1974), Springer-Verlag: Springer-Verlag Berlin and New York) · Zbl 0295.49001 |
| [32] | Bertram, J. E.; Sarachik, P. E., Stability of circuits with randomly time-varying parameters, IRE Transactions on Circuit Theory CT-6, 809-823 (1959) |
| [33] | Bertsekas, D., Approximate policy iteration: a survey and some new methods, Journal of Control Theory, 9, 310-335 (2011) · Zbl 1249.90179 |
| [34] | Bertsekas, D. P., Dynamic programming and optimal control (1995), Athena Scientific: Athena Scientific Belmont, MA · Zbl 0904.90170 |
| [35] | Bertsekas, D. P.; Shreve, S. E., Dynamic programming and stochastic control (1978), Academic Press: Academic Press New York |
| [36] | Bhatt, A. G.; Borkar, V. S., Occupation measures for controlled Markov processes: characterization and optimality, The Annals of Probability, 24, 1531-1562 (1996) · Zbl 0863.93086 |
| [37] | Bhatt, A. G.; Budhiraja, A.; Karandikar, R. L., Markov property and ergodicity of the nonlinear filter (2000) · Zbl 0970.60047 |
| [38] | Bhatt, A. G.; Karandikar, R. L., Invariant measures and evolution equations for Markov processes characterized via martingale problems, The Annals of Probability, 21, 2246-2268 (1993) · Zbl 0790.60062 |
| [39] | Bismut, J.-M., Conjugate convex functions in optimal stochastic control, Journal of Mathematical Analysis and Applications, 44, 384-404 (1973) · Zbl 0276.93060 |
| [40] | Bismut, J.-M., Théorie probabiliste du contrôle des diffusions, Memoirs of the American Mathematical Society, 167 (1976) · Zbl 0323.93046 |
| [41] | Bismut, J.-M., Duality methods in the control of densities, SIAM Journal on Control and Optimization, 16, 771-777 (1978) · Zbl 0393.49013 |
| [42] | Bismut, J.-M., An introductory approach to duality in stochastic control, SIAM Review, 20, 62-78 (1978) · Zbl 0378.93049 |
| [43] | Bismut, J.-M., Partially observed diffusions and their control, SIAM Journal on Control and Optimization, 20, 302-309 (1982) · Zbl 0486.93076 |
| [44] | Bismut, J.-M.; Michel, D., Diffusions conditionnelles, Journal of Functional Analysis, 44, 174-211 (1982) · Zbl 0475.60061 |
| [45] | Blankenship, G.; Papanicolaou, G. C., Stability and control of systems with wide band noise disturbances, SIAM Journal of Applied Mathematics, 34, 437-476 (1978) · Zbl 0392.93040 |
| [46] | Borkar, V. S., The Nisio semigroup for controlled diffusions with partial observations, Stochastics, 17, 145-160 (1986) · Zbl 0596.93068 |
| [47] | Borkar, V. S., Optimal control of diffusion processes (1989), Longman Scientific and Technical: Longman Scientific and Technical Harlow, Essex, UK · Zbl 0669.93065 |
| [48] | Borkar, V. S., On ergodic control of degenerate diffusions, Journal of Optimization Theory and Applications, 86, 251-261 (1995) · Zbl 0835.49012 |
| [49] | Borkar, V. S.; Gaitsgory, V., Averaging of singularly perturbed controlled stochastic differential equations, Applied Mathematics and Optimization, 56, 169-209 (2007) · Zbl 1139.93022 |
| [50] | Borkar, V. S.; Ghosh, M. K., Ergodic control of multidimensional diffusions: I, the existence results, SIAM Journal on Control and Optimization, 26, 112-126 (1988) · Zbl 0646.93068 |
| [51] | Borkar, V. S.; Ghosh, M. K., Ergodic control of partially degenerate diffusions in a compact domain, Stochastics and Stochastics Reports, 75, 221-231 (2003) · Zbl 1046.93048 |
| [52] | Buche, R.; Kushner, H., Rate of convergence for constrained stochastic approximation algorithms, SIAM. J. Control Optim., 40, 1011-1041 (2001) · Zbl 1011.62082 |
| [53] | Buche, R.; Kushner, H. J., Control of mobile communication systems with time-varying channels via stability methods, IEEE Transactions on Automatic Control, 49, 1954-1962 (2004) · Zbl 1365.90049 |
| [54] | Bucy, R. S., Stability and positive supermartingales, Journal of Differential Equations, 1, 151-155 (1965) · Zbl 0203.17505 |
| [55] | Budhiraja, A., Asymptotic stability, ergodic and other asymptotic properties of the nonlinear filter, Annales de l’Institut Henri Poincaré; Probabilités et Statistiques, 39, 919-941 (2003) · Zbl 1160.60318 |
| [56] | Budhiraja, A.; Dupuis, P., Simple necessary and sufficient conditions for the stability of constrained processes, SIAM Journal on Applied Mathematics, 1686-1700 (1999) · Zbl 0934.93068 |
| [57] | Budhiraja, A.; Kushner, H. J., Approximation and limit results for nonlinear filters over an infinite time interval, SIAM Journal on Control and Optimization, 1946-1979 (1999) · Zbl 0934.93064 |
| [58] | Budhiraja, A.; Kushner, H. J., Robustness of nonlinear filters over the infinite time interval, SIAM Journal on Control and Optimization, 37, 1618-1637 (1999) · Zbl 0915.93063 |
| [59] | Budhiraja, A.; Kushner, H. J., Approximation and limit results for nonlinear filters over an infinite time interval: Part II, random sampling algorithms, SIAM Journal on Control and Optimization, 38, 1874-1908 (2000) · Zbl 1031.93148 |
| [60] | Budhiraja, A.; Kushner, H. J., Monte Carlo algorithms and asymptotic problems in nonlinear filtering, (Stochastics in finite and infinite dimensions (2000), Birkhäuser), 59-87 · Zbl 1129.93535 |
| [61] | Budhiraja, A.; Kushner, H. J., A nonlinear filtering algorithm based on an approximation of the conditional distribution, IEEE Transactions on Automatic Control, 45, 580-585 (2000) · Zbl 0976.93080 |
| [62] | Canon, M. D.; Cullum, C. D.; Polak, E., The theory of optimal control and mathematical programming (1970), McGraw-Hill: McGraw-Hill New York · Zbl 0264.49001 |
| [63] | Christopeit, N., Existence of optimal controls under partial observation, Probability Theory and Related Fields, 51, 201-213 (1980) · Zbl 0407.93064 |
| [64] | Clark, J. M.C., The design of robust approximations to the stochastic differential equations of nonlinear filtering, (Skwirzynski, J., Communication systems and random process theory (1978), Sijtoff and Noordhoff: Sijtoff and Noordhoff Alphen aan der Rihn, Amsterdam) |
| [65] | Cox, R. M.; Karatzas, I., Stationary control of Brownian motion in several dimensions, Advances in Applied Probability, 17, 531-561 (1985) · Zbl 0574.93068 |
| [66] | Crandall, M. G.; Ishii, H.; Lions, P.-L., User’s guide to viscosity solutions of second order partial differential equations, American Mathematical Society. Bulletin, 27, 1-67 (1992) · Zbl 0755.35015 |
| [67] | Crandall, M. G.; Lions, P.-L., Viscosity solutions of Hamilton-Jacobi equations, Transactions of the American Mathematical Society, 277, 1-42 (1983) · Zbl 0599.35024 |
| [68] | Crisan, D.; Rozovskii˘, B., The oxford handbook of nonlinear filtering (2011), Oxford University Press: Oxford University Press Oxford · Zbl 1210.60005 |
| [69] | Curtain, R. F.; Ichikawa, A., The separation principle for stochastic evolution equations, SIAM Journal on Control and Optimization, 15, 367-383 (1977) · Zbl 0359.60076 |
| [70] | Davis, M. H.A., On the existence of optimal policies in stochastic control, SIAM Journal on Control and Optimization, 11, 587-594 (1973) · Zbl 0238.93044 |
| [71] | Davis, M. H.A., Nonlinear semigroups in the control of partially-observable stochastic systems, (Measure theory applications to stochastic analysis. Measure theory applications to stochastic analysis, Lect. notes in math., Vol. 695 (1977), Springer-Verlag: Springer-Verlag Berlin and New York), 39-47 · Zbl 0392.60064 |
| [72] | Davis, M. H.A., A pathwise solution to the equations of nonlinear filtering, Teoriya Veroyatnostei i ee Primeneniya, 27, 160-167 (1982) · Zbl 0487.60049 |
| [73] | Davis, M. H.A.; Varaiya, P., Dynamic programming conditions for partially observable stochastic systems, SIAM Journal on Control, 11, 226-261 (1973) · Zbl 0258.93029 |
| [74] | Delyon, B.; Zeitouni, O., Lyapunov exponents for filtering problems, (Davis, M. H.A.; Elliot, R. J., Applied stochastic analysis (1991)), 511-521 · Zbl 0738.60033 |
| [75] | Derman, C., On sequential decisions and Markov chains, Managment Science, 9, 16-24 (1962) · Zbl 0995.90621 |
| [76] | DiMasi, G. B.; Runggaldier, W. J., On robust approximation in nonlinear filtering, (Kohlmann, M.; Christopeit, N., Stochastic Differential Systems. Stochastic Differential Systems, Lecture notes in cont. and inform. sci., Vol. 43 (1982), Springer-Verlag: Springer-Verlag Berlin and New York), 179-186 · Zbl 0502.93066 |
| [77] | Doob, J. L., Stochastic processes (1953), Wiley: Wiley New York · Zbl 0053.26802 |
| [78] | Doucet, A.; Johanson, J., A tutorial on particle filtering and smoothing: fifteen years later, (Crisan, D.; Rozovskii˘, B., The oxford handbook of nonlinear filtering (2011), Oxford University Press: Oxford University Press Oxford) · Zbl 1513.60043 |
| [79] | Duncan, T. E.; Pasik-Duncan, B.; Stettner, L., On ergodic control of stochastic evolution equations, Stochastic Analysis and Applications, 15, 523-570 (1997) · Zbl 0894.60056 |
| [80] | Duncan, T. E.; Varaiya, P., On the solutions of a stochastic control problem, SIAM Journal on Control, 9, 35-371 (1971) · Zbl 0204.47303 |
| [81] | Dupuis, P.; Ellis, R. S., A weak convergence approach to the theory of large deviations (1997), Wiley: Wiley New York · Zbl 0904.60001 |
| [82] | Dupuis, P.; McEneaney, W. M., Risk-sensitive and robust escape criteria, SIAM Journal on Control and Optimization, 35, 2021-2049 (1997) · Zbl 0889.35014 |
| [83] | Dupuis, P.; Williams, R. J., Lyapunov functions for semimartingale reflecting Brownian motions, The Annals of Probability, 22, 680-702 (1994) · Zbl 0808.60068 |
| [84] | Dynkin, E. B., Markov processes (1965), Springer-Verlag: Springer-Verlag Berlin and New York · Zbl 0132.37901 |
| [85] | Echeverria, P. E., A criterion for invariant measures of Markov processes, Zeitschrift fur Wahrscheinlichkeitstheorie und Verwandte Gebiete, 61, 1-16 (1982) · Zbl 0476.60074 |
| [86] | El Karoui, N.; Lepeltier, J.-P.; Marchal, B., Nisio semigroup associated to the control of Markov processes, (Stochastic differential systems. Stochastic differential systems, Lect. notes in cont. inf. sci., Vol. 43 (1982), Springer-Verlag: Springer-Verlag Berlin and New York), 294-301 · Zbl 0503.93072 |
| [87] | El Karoui, N.; Nguyen, D. H.; Jeanblanc-Picqué, M., Compactification methods in the control of degenerate diffusions: existence of an optimal control, Stochastics, 20, 169-219 (1987) · Zbl 0613.60051 |
| [88] | Elliott, R. J., Stochastic calculus and applications (1982), Springer-Verlag: Springer-Verlag Berlin and New York · Zbl 0503.60062 |
| [89] | Elliott, R. J., Viscosity solutions and optimal control (1987), Pitman: Pitman Harlow, Essex UK · Zbl 0713.49035 |
| [90] | Elliott, R. J.; Kohlmann, M., The second order minimum principle and stochastic optimal control, Stochastics and Stochastics Reports, 46, 25-39 (1994) · Zbl 0824.60051 |
| [91] | Ethier, S. N.; Kurtz, T. G., Markov processes: characterization and convergence (1986), Wiley: Wiley New York · Zbl 0592.60049 |
| [92] | Fisk, D. L., Quasimartingales and stochastic integrals, Tech report 1, Math. Dept., Mich. State Univ. (1963) |
| [93] | Fleming, W. H., Some Markovian optimization problems, Journal of Mathematics and Mechanics, 12, 131-140 (1963) · Zbl 0128.37902 |
| [94] | Fleming, W. H., Duality and a priori estimates in Markovian optimization problems, Journal of Mathematical Analysis and Applications, 16, 254-279 (1966) · Zbl 0203.22101 |
| [95] | Fleming, W. H., Optimal continuous-parameter stochastic control, SIAM Review, 11, 470-505 (1969) · Zbl 0192.52501 |
| [96] | Fleming, W. H., Generalized solutions in optimal stochastic control, (Liu, P.; Roxin, E.; Sternberg, R., Differential games and control theory: III (1977), Marcel Dekker), 147-165 · Zbl 0367.93029 |
| [97] | Fleming, W. H., Nonlinear semigroup for controlled partially observed diffusions, SIAM Journal on Control and Optimization, 20, 286-301 (1982) · Zbl 0479.93076 |
| [98] | Fleming, W. H.; McEneaney, W. M., Risk sensitive optimal control and differential games, (Stochastic theory and adaptive control. Stochastic theory and adaptive control, Lecture notes in cont and inf. sci., Vol. 184 (1992), Springer-Verlag: Springer-Verlag Berlin and New York), 185-197 · Zbl 0788.90097 |
| [99] | Fleming, W. H.; Nisio, M., On the existence of optimal controls, Journal of Mathematics and Mechanics, 15, 777-794 (1966) · Zbl 0144.39901 |
| [100] | Fleming, W. H.; Pardoux, E., Optimal control for partially observed systems, SIAM Journal on Control and Optimization, 20, 261-285 (1982) · Zbl 0484.93077 |
| [101] | Fleming, W. H.; Rishel, R., Deterministic and stochastic optimal control (1975), Springer-Verlag: Springer-Verlag Berlin and New York · Zbl 0323.49001 |
| [102] | Fleming, W. H.; Soner, H. M., Controlled Markov processes and viscosity solutions (2006), Springer-Verlag: Springer-Verlag New York · Zbl 1105.60005 |
| [103] | Fleming, W. H.; Vermes, D., Convex duality approach to the optimal control of diffusions, SIAM Journal on Control and Optimization, 27, 1136-1155 (1989) · Zbl 0693.93082 |
| [104] | Florentin, J. J., Optimal control of continuous time, Markov, stochastic systems, Journal of Electronics and Control, 10, 473-488 (1961) |
| [105] | Framstad, N. C.; Øksendal, B.; Sulem, A., Sufficient stochastic maximum principle for the optimal control of jump diffusions and applications to finance, Journal of Optimization Theory and Applications, 121, 77-98 (2004) · Zbl 1140.93496 |
| [106] | Fujisaki, H.; Kallianpur, G.; Kunita, H., Stochastic differential equations for the nonlinear filtering problem, Osaka Mathematical Journal, 9, 19-40 (1972) · Zbl 0242.93051 |
| [107] | Fujita, Y.; Morimoto, H., Ergodic control of stochastic differential equations with controller constraints, Stochastics and Stochastics Reports, 58, 245-257 (1996) · Zbl 0867.93084 |
| [108] | Gaitsgory, V.; Quincampoix, M., Linear programming approach to deterministic infinite-horizon optimal control problems with discounting, SIAM Journal on Control and Optimization, 48, 2480-2512 (2009) · Zbl 1201.49040 |
| [109] | Girsanov, I. V., On transforming a certain class of stochastic processes by absolutely continuous transformation of measure, Theory of Probability and Its Applications, 5, 285-301 (1960) · Zbl 0100.34004 |
| [110] | Hale, J. K., Sufficient conditions for stability and instability of autonomous functional differential equations, Journal of Differential Equations, 1, 452-482 (1965) · Zbl 0135.30301 |
| [111] | Halkin, H., A maximum principle of the Pontriagin type for systems described by nonlinear difference equations, SIAM Journal on Control, 4, 90-111 (1966) · Zbl 0152.09301 |
| [112] | Haussmann, U., Some examples of optimal stochastic controls or: The stochastic maximum principle at work, SIAM Review, 23, 292-307 (1981) · Zbl 0466.93073 |
| [113] | Haussmann, U. G., Existence of partially observable stochastic optimal controls, (Arató, M.; Vermes, D.; Balakrishnan, A. V., Stochastic differential systems. Stochastic differential systems, Lecture notes in cont. and inform. sci, Vol. 36 (1981), Springer-Verlag: Springer-Verlag Berlin and New York), 79-84 · Zbl 0479.93036 |
| [114] | Haussmann, U. G., Existence of optimal Markovian controls for degenerate diffusions, (Christopeit, N.; Helmes, K.; Kohlmann, M., Stochastic differential systems. Stochastic differential systems, Lecture notes on cont. and inform. sci., 78 (1986), Springer-Verlag: Springer-Verlag Berlin and New York), 171-186 · Zbl 0593.93068 |
| [115] | Haussmann, U. G., A stochastic maximum principle for optimal control of diffusions (1986), Longman Scientific and Technical: Longman Scientific and Technical Harlow, Essex, UK · Zbl 0616.93076 |
| [116] | Haussmann, U. G.; Lepeltier, J. P., On the existence of optimal controls, SIAM Journal on Control and Optimization, 28, 851-902 (1990) · Zbl 0712.49013 |
| [117] | Helmes, K., Numerical methods for optimal stopping using linear and nonlinear programming, (Stochastic theory and control. Stochastic theory and control, Lect. notes in cont and inf. sci, Vol. 280 (2002), Springer-Verlag: Springer-Verlag Berlin and New York), 185-203 · Zbl 1036.60033 |
| [118] | Helmes, K.; Stockbridge, R. H., Linear programming approach to the optimal stopping of singular stochastic processes, Stochastic and Stochastics Reports, 35, 309-335 (2007) · Zbl 1110.60038 |
| [119] | Hernández-Lerma, O., Average cost Markov decision processes: Optimality conditions, Journal of Mathematical Analysis and Applications, 158, 396-406 (1991) · Zbl 0739.90072 |
| [120] | Ho, Y.-C.; Bryson, A. E., Applied optimal control (1969), Blaisdell: Blaisdell Waltham |
| [121] | Holtzman, J., On the maximum principle for nonlinear discrete-time systems, IEEE Transactions on Automatic Control, 4, 528-547 (1966) |
| [122] | Ikeda, N.; Watanabe, S., Stochastic differential equations and diffusion processes (1989), North-Holland: North-Holland Amsterdam · Zbl 0684.60040 |
| [123] | Infante, E. F., On the stability of some linear nonautonomous random systems, Transactions of the ASME. Series E, Journal of Applied Mechanics, 35, 7-12 (1968) · Zbl 0155.13001 |
| [124] | Ito, K.; Xiong, K., Gaussian filters for nonlinear filtering problems, IEEE Transactions on Automatic Control, 45, 910-927 (2000) · Zbl 0976.93079 |
| [125] | Kaczmarek, P.; Kent, S.; Rus, G.; Stockbridge, R.; Wade, B., Numerical solution of a long-term average control problem for singular stochastic processes, Mathematical Methods of Operations Research, 66, 451-473 (2007) · Zbl 1125.93071 |
| [126] | Kailath, T., An innovations approach to least squares filtering, IEEE Transactions on Automatic Control, 13, 646-655 (1968) |
| [127] | Kalman, R. E., Contributions to the theory of optimal control, Sociedad Matemática Mexicana. Boletín., 5, 102-119 (1960) · Zbl 0112.06303 |
| [128] | Kalman, R. E., A new approach to linear filtering and prediction problems, Journal of Basic Engineering, 82, 35-45 (1960) |
| [129] | Kalman, R. E.; Bucy, R. S., New results in linear filtering and prediction theory, Journal of Basic Engineering, 83, 95-108 (1961) |
| [130] | Karatzas, I.; Lehoczky, J.; Sethi, S.; Shreve, S., Expicit solution of a general consumption-investment problem, Mathematics of Operations Research, 11, 261-294 (1986) · Zbl 0622.90018 |
| [131] | Karatzas, I.; Shreve, S. E., Brownian motion and stochastic calculus (1988), Springer-Verlag: Springer-Verlag New York · Zbl 0638.60065 |
| [132] | Karatzas, I.; Shreve, S. E., Methods of mathematical finance (1998), Springer-Verlag: Springer-Verlag Berlin and New York · Zbl 0941.91032 |
| [133] | Kats, I. Y., On the stability of stochastic systems in the large, Journal of Applied Mathematics and Mechanics. Journal of Applied Mathematics and Mechanics, Prikl. Mat. Meh., 28, 366-372 (1964), Translation of · Zbl 0139.34304 |
| [134] | Kats, I. Y.; Krasovskii, N., On the stability of systems with random parameters, Journal of Applied Mathematics and Mechanics. Journal of Applied Mathematics and Mechanics, Prikl. Mat. Meh., 24, 809-823 (1960), Translation of · Zbl 0103.36403 |
| [135] | Khasminskii, R. Z., Ergodic properties of recurrent diffusion processes and stabilization of the solution to the Cauchy problem for parabolic equations, Theory of Probability and Its Applications, 5, 179-196 (1960) · Zbl 0106.12001 |
| [136] | Khasminskii, R. Z., On the stability of the trajectory of Markov processes, Journal of Applied Mathematics and Mechanics, 26, 1554-1565 (1962), Translated from Prikl. Mat. Meh., 26, 1025-1032 · Zbl 0137.35806 |
| [137] | Khasminskii, R. Z., A limit theorem for the solutions of differential equations with random right- hand sides, Theory of Probability and Its Applications, 11, 390-406 (1966) |
| [138] | Khasminskii, R. Z., On the stability of nonlinear stochastic systems, Journal of Applied Mathematics and Mechanics, 30, 1082-1089 (1967), Translated from, Prikl. Mat. Meh., 30, 915-921 · Zbl 0157.24603 |
| [139] | Khasminskii, R. Z., Stochastic stability of differential equations (1980), Sijthoff, Noordhoff: Sijthoff, Noordhoff Amsterdam,Alphen aan den Rijn, Translation of the 1967 Russian book · Zbl 1259.60058 |
| [140] | Khasminskii, R. Z., Stochastic stability of differential equations (2011), Springer-Verlag: Springer-Verlag Berlin and New York · Zbl 1259.60058 |
| [141] | Kleptsyna, M.; Veretennikov, A. Y., On continuous-time ergodic filters with wrong initial data, Theory of Probability and Its Applications, 53, 240-276 (2008) |
| [142] | Kohlmann, M., Survey on existence results in nonlinear optimal stochastic control of semimartingales, (Lecture notes in cont. and inform. sci., Vol. 30 (1981), Springer-Verlag: Springer-Verlag Berlin and New York), 187-210 · Zbl 0488.93061 |
| [143] | Kohlmann, M., Existence of optimal controls for a partially observed semimartingale, Stochastic Processes and their Applications, 13, 215-226 (1982) · Zbl 0495.60009 |
| [144] | Kozin, F., A survey of stability of stochastic systems, Automatica, 5, 95-112 (1969) · Zbl 0164.40301 |
| [145] | Krasovskii, N., Stability of motion (1963), Stanford University Press.: Stanford University Press. Stanford CA, Translation of the 1963 book in Russian · Zbl 0109.06001 |
| [146] | Krylov, N. V., Controlled diffusion processes (1980), Springer-Verlag: Springer-Verlag Berlin and New York · Zbl 0459.93002 |
| [147] | Krylov, N. V., On the equivalence of \(\sigma \)-algebras in the filtering of diffusion processes, Theory of Probability and Its Applications, 24, 771-781 (1980) · Zbl 0441.60042 |
| [148] | Krylov, N. V., Filtering equations for partially observable diffusion processes with Lipschitz continuous coefficients, (Crisan, D.; Rozovskii˘, B., The Oxford handbook of nonlinear filtering (2011), Oxford University Press: Oxford University Press Oxford), 169-194 · Zbl 1223.93112 |
| [149] | Krylov, N. V.; Rozovskii˘, B. L., The Cauchy problem for linear stochastic partial differential equations, Izvestiya Akademii Nauk SSSR. Seriya Matematicheskaya, 41, 1329-1347 (1977) · Zbl 0371.60076 |
| [150] | Kunita, H., Nonlinear filtering problems II: Associated equations, (Crisan, D.; Rozovskii˘, B., The Oxford handbook of nonlinear filtering (2011), Oxford University Press: Oxford University Press Oxford), 55-94 · Zbl 1223.93114 |
| [151] | Kurtz, T. G., A limit theorem for perturbed operator semigroups with applications to random evolutions, Journal of Functional Analysis, 12, 55-67 (1973) · Zbl 0246.47053 |
| [152] | Kurtz, T. G., Semigroups of conditional shifts and approximation of Markov processes, The Annals of Probability, 3, 618-642 (1975) · Zbl 0318.60026 |
| [153] | Kurtz, T. G., Martingale problems for controlled processes, (Germani, A., Stochastic modeling and filtering. Stochastic modeling and filtering, Lecture notes in cont. and inform. sci, Vol. 91 (1987), Springer-Verlag: Springer-Verlag Berlin and New York), 75-90 · Zbl 0642.93065 |
| [154] | Kurtz, T. G.; Ocone, D. L., Unique characterization of conditional distributions in nonlinear filtering, The Annals of Probability, 16, 80-107 (1988) · Zbl 0655.60035 |
| [155] | Kurtz, T. G.; Pardoux, E.; Protter, P., Stratonovich stochastic differential equations driven by semimartingales, Annales de l’Inst. Henri Poincare, Section B, 31, 351-377 (1995) · Zbl 0823.60046 |
| [156] | Kurtz, T. G.; Protter, P., Weak limit theorems for stochastic integrals and stochastic differential equations, The Annals of Probability, 1035-1070 (1991) · Zbl 0742.60053 |
| [157] | Kurtz, T. G.; Stockbridge, R. H., Existence of Markov controls and characterization of optimal Markov controls, SIAM Journal on Control and Optimizatio, 36, 609-653 (1998) · Zbl 0935.93064 |
| [158] | Kurzweil, J., On the inversion of Liapunov’s second theorem on stability of motion, Amer. Math. Soc. Translations, Ser 2, 24, 19-77 (1963) · Zbl 0127.30703 |
| [159] | Kushner, H. J., On the differential equations satisfied by conditional probability densities of Markov processes, SIAM Journal on Control, 2, 106-119 (1962) · Zbl 0131.16602 |
| [160] | Kushner, H. J., Optimal stochastic control, IRE Transactions on Automatic Control, 7 (1962) · Zbl 0111.33001 |
| [161] | Kushner, H. J., On the stability of stochastic dynamical systems, Proceedings of the National Academy of Sciences, 53, 5-12 (1965) · Zbl 0143.19005 |
| [162] | Kushner, H. J., On the stochastic maximum principle: fixed time of control, Journal of Mathematical Analysis and Applications, 11, 78-92 (1965) · Zbl 0142.06802 |
| [163] | Kushner, H. J., On the theory of stochastic stability, (Advances in control systems, Vol. 4 (1965), Academic Press: Academic Press New York), 73-102 |
| [164] | Kushner, H. J., Some problems and recent results in stochastic control, (Proc. IEEE internat. conf (1965), IEEE Press: IEEE Press New York) |
| [165] | Kushner, H. J., On the existence of optimal stochastic controls, SIAM Journal on Control and Optimization, 3, 463-474 (1966) · Zbl 0147.16103 |
| [166] | Kushner, H. J., Sufficient conditions for the optimality of a stochastic control, SIAM Journal on Control, 3, 499-508 (1966) · Zbl 0138.11104 |
| [167] | Kushner, H. J., Approximations to optimal nonlinear filters, IEEE Transactions on Automatic Control, AC-12, 546-556 (1967) |
| [168] | Kushner, H. J., Converse theorems for stochastic Liapunov functions, SIAM Journal on Control and Optimization, 5, 228-233 (1967) · Zbl 0183.19401 |
| [169] | Kushner, H. J., Stochastic stability and control (1967), Academic Press: Academic Press New York · Zbl 0244.93065 |
| [170] | Kushner, H. J., The concept of invariant set for stochastic dynamical systems and applications to stochastic stability, (Karreman, H., Stochastic optimization and control (1968), John Wiley and Sons: John Wiley and Sons New York) · Zbl 0251.93034 |
| [171] | Kushner, H. J., On the stability of stochastic differential-difference equations, Journal of Differential Equations, 4, 424-443 (1968) · Zbl 0169.11601 |
| [172] | Kushner, H. J., The Cauchy problem for a class of degenerate parabolic equations and asymptotic properties of the related diffusion process, Journal of Differential Equations, 6, 209-231 (1969) · Zbl 0181.44305 |
| [173] | Kushner, H. J., Invariance theorems of stochastic stability, and applications to identification and adaptation for linear systems, (Proc. 1972 CDC (1972), IEEE Press: IEEE Press New York) |
| [174] | Kushner, H. J., Necessary conditions for continuous parameter stochastic optimization problems, SIAM Journal on Control and Optimization, 10, 550-565 (1972) · Zbl 0242.93063 |
| [175] | Kushner, H. J., Necessary conditions for discrete parameter stochastic optimization problems, (LeCam, L.; Neyman, J.; Scott, E., Proceedings of the sixth Berkeley symposium on mathematical statistics and probability, vol. III (1972), Univ. Of California Press), 667-686 · Zbl 0275.93062 |
| [176] | Kushner, H. J., Stability of stochastic dynamical systems, (Curtain, R., Lecture notes in math, Vol. 294 (1972), Springer-Verlag: Springer-Verlag Berlin and New York), 97-124 · Zbl 0275.93055 |
| [177] | Kushner, H. J., Stochastic stability, (Curtain, R., Stability of stochastic dynamical systems. Stability of stochastic dynamical systems, Lecture notes in math., 294 (1972), Springer-Verlag: Springer-Verlag Berlin and New York), 97-124 · Zbl 0275.93055 |
| [178] | Kushner, H. J., Existence results for optimal stochastic controls, Journal of Optimization Theory and Applications, 15, 347-359 (1975) · Zbl 0279.93056 |
| [179] | Kushner, H. J., Probability methods for approximations in stochastic control and for elliptic equations (1977), Academic Press: Academic Press New York · Zbl 0547.93076 |
| [180] | Kushner, H. J., Optimality conditions for the average cost per unit time problem with a diffusion model, SIAM Journal on Control and Optimization, 16, 330-346 (1978) · Zbl 0376.93044 |
| [181] | Kushner, H. J., Jump-diffusion approximations for ordinary differential equations with wideband random right hand sides, SIAM Journal on Control and Optimization, 17, 729-744 (1979) · Zbl 0432.60096 |
| [182] | Kushner, H. J., A robust discrete state approximation to the optimal nonlinear filter for a diffusion, Stochastics and Stochastics Reports, 3, 75-83 (1979) · Zbl 0421.60054 |
| [183] | Kushner, H. J., Approximate invariant measures for the asymptotic distributions of differential equations with wide band right hand sides, (Fleming, W.; Gorostiza, L., Advances in filtering and optimal stochastic control. Advances in filtering and optimal stochastic control, Lecture notes in contr. and inform. sci., Vol. 42 (1982), Springer-Verlag: Springer-Verlag Berlin and New York), 192-198 · Zbl 0498.93055 |
| [184] | Kushner, H. J., Asymptotic distributions of solutions of ordinary differential equations with wide band noise inputs; approximate invariant measures, Stochastics, 6, 259-278 (1982) · Zbl 0483.60049 |
| [185] | Kushner, H. J., Approximation and weak convergence methods for random processes with applications to stochastic systems theory (1984), MIT Press: MIT Press Cambridge, Mass · Zbl 0551.60056 |
| [186] | Kushner, H. J., Almost optimal controls for wide band width noise driven systems, (Fleming, W.; Lions, P.-L., IMA volumes in mathematics and applications. IMA volumes in mathematics and applications, Stochastic differential systems, stochastic control theory and applications., Vol. 10 (1987), Springer-Verlag: Springer-Verlag Berlin and New York), 255-274 |
| [187] | Kushner, H. J., Filtering and control for wide bandwidth noise driven systems, IEEE Transactions on Automatic Control, 32, 123-133 (1987) · Zbl 0617.93060 |
| [188] | Kushner, H. J., Approximations and optimal controls for the pathwise average cost per unit time and discounted problems for wideband noise driven systems, SIAM Journal on Control and Optimization, 27, 546-562 (1989) · Zbl 0693.93083 |
| [189] | Kushner, H. J., (Weak convergence methods and singularly perturbed stochastic control and filtering problems. Weak convergence methods and singularly perturbed stochastic control and filtering problems, Systems and control., Vol. 3 (1990), Birkhäuser: Birkhäuser Boston) · Zbl 0931.93003 |
| [190] | Kushner, H. J., Functional occupation measures and ergodic cost problems for singularly perturbed stochastic systems, (Applied stochastic analysis: stochastics monog. 5 (1991), Gordon and Breach: Gordon and Breach London), 79-99 · Zbl 0760.93085 |
| [191] | Kushner, H. J., Nonlinear filtering for singularly perturbed systems, (Mayer-Wolfe, E.; Merzbach, E.; Shwartz, A., Stochastic analysis (1991), Academic Press: Academic Press New York), 347-369 · Zbl 0737.60034 |
| [192] | Kushner, H. J., Existence of optimal controls for variance control, (McEneaney, W.; Yin, G.; Zhang, Q., Stochastic analysis, control, optimization and applications: a volume in honor of W.H. Fleming (1998), Birkhäuser: Birkhäuser Boston) |
| [193] | Kushner, H. J., Heavy traffic analysis of controlled queueing and communication networks (2001), Springer-Verlag: Springer-Verlag Berlin and New York · Zbl 0988.90004 |
| [194] | Kushner, H. J., Numerical approximations for stochastic differential games, SIAM Journal on Control and Optimization, 457-486 (2002) · Zbl 1014.60035 |
| [195] | Kushner, H. J., Numerical approximations for stochastic differential games: The ergodic case, SIAM Journal on Control and Optimization, 42, 1911-1933 (2004) · Zbl 1151.91352 |
| [196] | Kushner, H. J., Control of multi-node mobile communications networks with time varying channels via stability methods, Queueing Systems, 54, 317-329 (2006) · Zbl 1103.93048 |
| [197] | Kushner, H. J., Numerical methods for controlled stochastic delay systems (2008), Birkhäuser: Birkhäuser Boston · Zbl 1219.93001 |
| [198] | Kushner, H. J.; Barnea, D., On the control of linear functional-differential equations with quadratic cost, SIAM Journal on Control and Optimization, 8, 257-272 (1970) · Zbl 0205.16901 |
| [199] | Kushner, H. J.; Clark, D., Stochastic approximation for constrained and unconstrained systems (1978), Springer-Verlag: Springer-Verlag Berlin and New York · Zbl 0381.60004 |
| [200] | Kushner, H. J.; Dupuis, P., Numerical methods for stochastic control problems in continuous time (2001), Springer-Verlag: Springer-Verlag Berlin and New York · Zbl 0968.93005 |
| [201] | Kushner, H. J.; Huang, H., Diffusion approximations for the analysis of digital phase locked loops, IEEE Transaction on Information Theory, 28, 384-390 (1982) · Zbl 0477.94002 |
| [202] | Kushner, H. J.; Huang, H., Limits for parabolic partial differential equations with wide band stochastic coefficients, and an application to filtering theory, Stochastics, 14, 115-148 (1985) · Zbl 0566.60061 |
| [203] | Kushner, H. J.; Huang, H., Weak convergence approximations for partial differential equations with stochastic coefficients, Stochastics, 15, 209-245 (1985) · Zbl 0579.60052 |
| [204] | Kushner, H. J.; Huang, H., Approximate and limit results for nonlinear filters with wide bandwidth observation noise, Stochastics and Stochastics Reports, 16, 65-96 (1986) · Zbl 0595.60046 |
| [205] | Kushner, H. J.; Huang, H., Asymptotic properties, stability and “near” stationarity of parabolic partial differential equations with wide bandwidth inputs, Stochastics, 111-136 (1986) · Zbl 0585.60060 |
| [206] | Kushner, H. J.; Ju, W., Diffusion approximations to digital phase locked loops with wide band inputs, Journal of Mathematical Analysis and Applications, 86, 518-541 (1982) · Zbl 0485.93061 |
| [207] | Kushner, H. J.; Kleinman, A., Numerical methods for the solution of degenerate nonlinear equations arising in optimal stochastic control theory, IEEE Transactions on Automatic Control, 13, 344-353 (1968) |
| [208] | Kushner, H. J.; Martins, L., Limit theorems for pathwise average cost per unit time problems for queues in heavy traffic, Stochastics and Stochastics Reports, 42, 25-51 (1993) · Zbl 0806.60019 |
| [209] | Kushner, H. J.; Ramachandran, K., Nearly optimal singular controls for wideband noise driven systems, SIAM Journal on Control and Optimization, 26, 561-591 (1988) · Zbl 0644.93069 |
| [210] | Kushner, H. J.; Runggaldier, W., Filtering and control for wide bandwidth noise and “nearly” linear systems, (Byrnes, C.; Kurzhanski, A., Modeling and adaptive control. Modeling and adaptive control, Lecture notes in cont. and infor. sci., Vol. 105 (1986), Springer-Verlag: Springer-Verlag Berlin and New York), 201-213 · Zbl 0663.93070 |
| [211] | Kushner, H. J.; Runggaldier, W., Nearly optimal state feedback controls for stochastic systems with wideband noise disturbances, SIAM Journal on Control and Optimization, 25, 298-315 (1987) · Zbl 0621.93079 |
| [212] | Kushner, H. J.; Yin, G., Stochastic approximation and recursive algorithms and applications (2003), Springer-Verlag: Springer-Verlag Berlin and New York · Zbl 1026.62084 |
| [213] | Kushner, H. J.; Yu, C. F., Approximations, existence and numerical procedures for optimal stochastic controls, Journal of Mathematical Analysis and Applications, 45, 563-587 (1974) · Zbl 0285.93037 |
| [214] | Kusuoka, S.; Stroock, D., The partial Malliavin calculus and its applications to nonlinear filtering, Stochastics, 12, 83-142 (1984) · Zbl 0567.60046 |
| [215] | LaSalle, J. P., The extent of asymptotic stability, Proceedings of the National Academy of Sciences, 46, 365 (1960) · Zbl 0094.28602 |
| [216] | LaSalle, J. P.; Lefschetz, S., Stability by Liapunov’s direct method with applications (1961), Academic Press: Academic Press New York · Zbl 0098.06102 |
| [217] | Lee, E. B.; Marcus, L., Foundations of optimal control theory (1967), Wiley: Wiley New York · Zbl 0159.13201 |
| [218] | Lefschetz, S., Stability of nonlinear control systems (1965), Academic Press: Academic Press London · Zbl 0136.08801 |
| [219] | Levine, W. S., The control handbook (1995), CRC Press: CRC Press Boca Raton |
| [220] | Lindquist, A., On feedback control of linear stochastic systems, SIAM Journal on Control, 11, 323-343 (1973) · Zbl 0256.93065 |
| [221] | Lions, P.-L., Optimal control of diffusion processes and Hamilton-Jacobi-Bellman equations. Part 3: regularity of the optimal cost function, (Nonlinear PDE and appl: college de france seminar, vol.V. Nonlinear PDE and appl: college de france seminar, vol.V, Research notes in math., vol. 93 (1982), Pitman: Pitman Boston) |
| [222] | Lions, P.-L., Optimal control of diffusion processes and Hamilton-Jacobi-Bellman equations: Part 1. the dynamic programming principle and applications, Communications in Partial Differential Equations, 8, 1101-1174 (1983) · Zbl 0716.49022 |
| [223] | Lions, P.-L., Optimal control of diffusion processes and Hamilton-Jacobi-Bellman equations. Part 2: Viscosity solutions and uniqueness, Communications in Partial Differential Equations, 8, 1220-1276 (1983) · Zbl 0716.49023 |
| [224] | Lions, P.-L.; Nisio, M., A uniqueness result for the semigroup associated with the Hamilton-Jacobi-Bellman operator, Proceedings of the Japan Academy, Series A,, 58, 273-276 (1982) · Zbl 0516.93065 |
| [225] | Liptser, R.; Shiryaev, A. N., Statistics of random processes (1977), Springer-Verlag: Springer-Verlag Berlin and New York · Zbl 0364.60004 |
| [226] | Ljung, L.; Söderström, T., Theory and practice of recursive identification (1983), MIT Press: MIT Press Cambridge, Mass · Zbl 0548.93075 |
| [227] | Luenberger, D., Observers for multivariable systems, IEEE Transactions on Automatic Control, 11, 190-197 (1966) |
| [228] | Mandl, P., On the control of nonterminating diffusions, Theory of Probability and Its Applications, 9, 591-603 (1964) · Zbl 0134.34403 |
| [229] | Manne, A., Linear programming and sequential decisions, Management Science, 6, 259-267 (1960) · Zbl 0995.90599 |
| [230] | Mao, X., Stability of stochastic differential equations with respect to semimartingales (1991), Longman Scientific and Technical: Longman Scientific and Technical Harlow, Essex, UK · Zbl 0724.60059 |
| [231] | Marcus, L., A brief history of control, (Elworthy, K.; Everitt, W.; Lee, E., Differential equations, dynamical systems and control sciences (pp. xxv-xl) (1994), Marcel Dekker: Marcel Dekker New York) · Zbl 0798.93002 |
| [232] | Massera, J. L., Contributions to stability theory, Annals of Mathematics, 2, 182-206 (1956) · Zbl 0070.31003 |
| [233] | Massera, J. L., Converse theorems of Liapunov’s second method, Sociedad Matemática Mexicana. Boletín., 5, 158-163 (1960) · Zbl 0102.30603 |
| [234] | McShane, E. J., On multipliers for Lagrange problems, American Journal of Mathematics, 61, 809-819 (1939) · Zbl 0022.23403 |
| [235] | McShane, E. J., Stochastic integrals and stochastic functional equations, SIAM Journal of Applied Mathematics, 17, 287-306 (1969) · Zbl 0182.51001 |
| [236] | McShane, E. J., Stochastic calculus and stochastic models (1974), Academic Press: Academic Press New York · Zbl 0292.60090 |
| [237] | McShane, E. J.; Warfield, R., On Fillopov’s implicit function theorem, Proceedings of the Americal Mathematical Society, 18, 41-47 (1967) · Zbl 0145.34403 |
| [238] | Menaldi, J.-L.; Robin, M., Ergodic control of reflected diffusions with jumps, Applied Mathematics and Optimization, 35, 117-137 (1997) · Zbl 0876.93095 |
| [239] | Merton, R., Optimal portfolio and consumption rules in continuous time, Journal of Economic Theory, 3, 373-413 (1972) · Zbl 1011.91502 |
| [240] | Meyn, S. P.; Tweedie, R. I., Markov chains and stochastic stability (1994), Springer-Verlag: Springer-Verlag Berlin and New York |
| [241] | Minorsky, N., Directional stability of automatically steered bodies, Journal of the American Society of Naval Engineers, 280-309 (1922) |
| [242] | Morozan, T., Stability of differential systems with random parameters, Journal of Mathematical Analysis and Applications, 24, 669-676 (1968) · Zbl 0227.93028 |
| [243] | Neustadt, L., An abstract variational theory with applications to a broad class of optimization problems, SIAM Journal on Control, 4, 505-527 (1966) · Zbl 0166.09401 |
| [244] | Nevelson, M.; Khasminskii, R., Stability of a linear system with random disturbances of its parameters, Journal of Applied Mathematics and Mechanics, 30, 487-493 (1966), Translation of, Prikl. Mat. Meh., 30, 404-409 (Russian) · Zbl 0153.20803 |
| [245] | Nisio, M., On a non-linear semi-group attached to stochastic optimal control, Publications of the Research Institute for Mathematical Sciences, 13, 513-537 (1976), Kyoto Univ · Zbl 0364.93039 |
| [246] | Nisio, M., On stochastic optimal control and envelope of Markovian semigroups, (Proc. Int. Symp. on stochastic differential equations (1976), Wiley: Wiley New York), 297-325 · Zbl 0418.49031 |
| [247] | Nisio, M., Lectures on stochastic control theory: ISI lecture series no. 9 (1981), MacMillan Co of India: MacMillan Co of India Delhi · Zbl 0453.93056 |
| [248] | Ocone, D.; Pardoux, E., Asymptotic stability of the optimal filter with respect to its initial condition, Siam Journal of Control and Optimization, 34, 226-243 (1996) · Zbl 1035.93508 |
| [249] | Papanicolaou, G. C., Asymptotic analysis of transport processes, American Mathematical Society. Bulletin, 81 (1975) · Zbl 0361.60056 |
| [250] | Papanicolaou, G. C.; Hersh, R., Some limit theorems for the solutions of differential equations with random right-hand sides, Indiana University Mathematics Journal, 21, 815-840 (1972) · Zbl 0222.60037 |
| [252] | Papanicolaou, G. C.; Varadhan, S. R.S., A limit theorem with strong mixing and two applications to stochastic differential equations, Communications on Pure and Applied Mathematics, 26, 497-524 (1973) · Zbl 0253.60065 |
| [253] | Pardoux, E., Stochastic partial differential equations and filtering of diffusion processes, Stochastics, 3, 127-167 (1979) · Zbl 0424.60067 |
| [254] | Peng, S., A general stochastic maximum principle for optimal control problems, SIAM Journal on Control and Optimizatio, 28, 966-979 (1990) · Zbl 0712.93067 |
| [255] | Pesch, H. J.; Bulirsch, R., The maximum principle, Bellman’s work and Caratheodory’s work, Journal of Optimization Theory and Applications, 80, 199-225 (1994) · Zbl 0798.49001 |
| [256] | Phillips, R. S.; Sarason, L., Elliptic-parabolic equations of the second order, Journal of Mathematics and Mechanics, 17, 891-917 (1968) · Zbl 0163.34402 |
| [257] | Pittel, B. G., A linear programming problem connected with optimal stationary control in a dynamic decision problem, Th. Prob. Appl, 16, 724-728 (1971) · Zbl 0265.60063 |
| [258] | Pontryagin, L.; Boltyanski, V.; Gamkrelidze, R.; Mischenko, E. F., Mathematical theory of optimal processes (1962), Wiley: Wiley New York · Zbl 0102.32001 |
| [259] | Protter, P., Stochastic integration and differential equations (1990), Springer: Springer Berlin and New York · Zbl 0694.60047 |
| [260] | Puterman, M. L., Markov decision processes (1994), Wiley: Wiley New York · Zbl 0829.90134 |
| [261] | Rishel, R. W., Necessary and sufficient dynamic programming conditions for continuous time stochastic optimal control, SIAM Journal on Control, 8, 559-571 (1970) · Zbl 0206.45804 |
| [262] | Rishel, R. W., Weak solutions of a partial differential equation on dynamic programming, SIAM Journal on Control, 9, 519-528 (1971) · Zbl 0209.17003 |
| [263] | Robin, M., Long-term average cost control problems for continuous time Markov processes: a survey, Acta Applicandae Mathematicae, 1, 281-299 (1983) · Zbl 0531.93068 |
| [264] | Rockafellar, R. T., Conjugate convex functions in optimal control and the calculus of variations, Journal of Mathematical Analysis and Applications, 32, 174-222 (1970) · Zbl 0218.49004 |
| [265] | Roxin, E., The existence of optimal controls, Michigan Mathematical Journal, 9, 109-119 (1962) · Zbl 0105.07801 |
| [266] | Rozovskii˘, B., A simple proof of uniqueness for the Kushner and Zakai equations, (Mayer-Wolfe, E.; Merzbach, E.; Shwartz, A., Stochastic analysis (1991), Academic Press: Academic Press New York), 449-458 · Zbl 0732.60055 |
| [267] | Rozovskii˘, B., Stochastic evolution systems (1990), Kluwer Academic Publishers: Kluwer Academic Publishers Amsterdam · Zbl 0724.60070 |
| [268] | Ruzicka, J., On the separation principle with bounded controls, Applied Mathematics and Optimization, 3, 242-261 (1976) · Zbl 0386.93056 |
| [269] | Shapley, L. S., Stochastic games, Proceedings of the National Academy of Sciences of the United States of America, 39, 1095-1100 (1953) · Zbl 0051.35805 |
| [270] | Shiryaev, A. N., Stochastic equations of nonlinear filtering of jump Markov processes, Problemy Peredachi Informatsii, 3, 3-22 (1966), see also Theory Probab. Appl., vol. 11, 1966, 179-184 · Zbl 0271.60082 |
| [271] | Shiryaev, A. N., Optimal stopping rules (1978), Springer-Verlag: Springer-Verlag Berlin and New York · Zbl 0391.60002 |
| [272] | Shreve, S. E., An introduction to singular stochastic control, (Stochastic differential systems, stochastic control theory and applications (1986), Springer: Springer New York), 513-528, iMA Vol. Math. Appl. 10, Minneapolis, Minn · Zbl 0850.93739 |
| [273] | Shreve, S. E.; Bertsekas, D. P., Alternative theoretical frameworks for finite horizon discrete-time stochastic optimal control, SIAM J Control Optim., 16, 953-978 (1978) · Zbl 0405.93044 |
| [274] | Stockbridge, R. H., Time-average control of a martingale problem: A linear programming formulation, The Annals of Probability, 18, 206-217 (1990) · Zbl 0699.49019 |
| [275] | Stockbridge, R. H., Time-average control of a martingale problem: existence of a stationary solution, The Annals of Probability, 18, 190-205 (1990) · Zbl 0699.49018 |
| [276] | Stratonovich, R. L., Conditional Markov processes, Th. Probab. Appl., 5, 159-178 (1960) · Zbl 0106.12401 |
| [277] | Stratonovich, R. L., Topics in the theory of random noise; volume 1 (1963), Gordon and Breach: Gordon and Breach New York and London, Translation of the 1961 book in Russian |
| [278] | Stratonovich, R. L., A new representation for stochastic integrals, SIAM Journal on Control. SIAM Journal on Control, Vestn. Most. Univ. ser. I. Mat. Mech, 4, 3-12 (1964), Translated from |
| [279] | Stroock, D. W.; Varadhan, S. R.S., Multidimensional diffusion processes (1979), Springer-Verlag: Springer-Verlag New York · Zbl 0426.60069 |
| [280] | Sussmann, H. J., On the gap between deterministic and stochastic ordinary differential equations, The Annals of Probability, 6, 19-41 (1978) · Zbl 0391.60056 |
| [281] | Tarres, R., Asymptotic evolution of a stochastic control problem, SIAM J. Cont. Optim., 23, 614-631 (1985) · Zbl 0579.93068 |
| [282] | Tsinias, J.; Spiliotis, J., A converse Liapunov theorem for robust exponential stability, (Aeyels, D.; Lamnabhi-Lagarrigue, F.; Van der Schaft, A., Lecture notes in cont. and inform. sci, vol. 246 (1999), Springer Verlag: Springer Verlag Berlin and New York), 355-374 · Zbl 0934.93067 |
| [283] | Twardowska, K., Wong-Zakai approximations for stochastic differential equations, Acta Applicandae Mathematicae, 43, 317-359 (1996) · Zbl 0860.60041 |
| [284] | Vinter, R.; Lewis, R., A necessary and sufficient condition for optimality of dynamic programming type, making no a priori assumptions on the controls, SIAM Journal on Control and Optimization, 16, 571-583 (1978) · Zbl 0392.49011 |
| [285] | Wald, A., Sequential decision functions (1950), Wiley: Wiley New York · Zbl 0036.09502 |
| [286] | Warfield, V., A stochastic maximum principle, SIAM Journal on Control and Optimizatio, 14, 803-826 (1976) · Zbl 0334.93057 |
| [287] | Whitt, W., Stochastic process limits (2001), Springer-Verlag: Springer-Verlag Berlin and New York |
| [288] | Whittle, P., Risk-sensitive optimal control (1990), Wiley: Wiley New York · Zbl 0718.93068 |
| [289] | Wong, E.; Zakai, M., On the relationship between ordinary and stochastic differential equations, International Journal of Engineering Science, 3, 213-229 (1985) · Zbl 0131.16401 |
| [290] | Wonham, W. M., Some applications of stochastic differential equations to optimal nonlinear filtering, SIAM Journal on Control, 2, 347-369 (1964) · Zbl 0143.19004 |
| [291] | Wonham, W. M., Liapunov criteria for weak stochastic stability, Journal of Differential Equations, 2, 195-207 (1966) · Zbl 0133.40601 |
| [292] | Wonham, W. M., A Liapunov method for the estimation of statistical averages, Journal of Differential Equations, 2, 365-377 (1966) · Zbl 0168.15903 |
| [293] | Wonham, W. M., On the separation theorem of stochastic control, SIAM Journal on Control, 6, 312-326 (1968) · Zbl 0164.19101 |
| [294] | Xiong, J., Particle approximations to the filtering problem in continuous time, (Crisan, D.; Rozovskii˘, B., The Oxford handbook of nonlinear filtering (2011), Oxford University Press: Oxford University Press Oxford), 635-655 · Zbl 1223.93117 |
| [295] | Yin, G., Stochastic approximation: theory and applications, (Kannan, D.; Lakshmikantham, V., Handbook of stochastic analysis (2002), Marcel Dekker: Marcel Dekker New York), 577-624 · Zbl 0992.62076 |
| [296] | Yong, J.; Zhou, X., Stochastic controls, hamiltonian systems and HJB equations (1999), Springer-Verlag: Springer-Verlag Berlin and New York · Zbl 0943.93002 |
| [297] | Young, L., Generalized curves and the existence of an attained absolute minimum in the calculus of variations, Comptes Rendus des Séances de la Société des Sciences et des Lettres de Varsovie CI III, 30, 212-234 (1937) · Zbl 0019.21901 |
| [298] | Zakai, M., On the ultimate boundedness of moments associated with solutions of stochastic differential equations, SIAM Journal on Control, 5, 588-593 (1967) · Zbl 0267.60066 |
| [299] | Zakai, M., A Liapunov criterion for the existence of stationary probability distributions of systems perturbed by noise, SIAM Journal on Control, 7, 390-397 (1969) · Zbl 0195.44505 |
| [300] | Zakai, M., On optimal filtering of diffusion processes, Zeitschrift fur Wahrscheinlichkeitstheorie und Verwandte Gebiete, 11, 230-243 (1969) · Zbl 0164.19201 |
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. In some cases that data have been complemented/enhanced by data from zbMATH Open. This attempts to reflect the references listed in the original paper as accurately as possible without claiming completeness or a perfect matching.
