The Markov chain approximation approach for numerical solution of stochastic control problems: experiences from Merton’s problem. (English) Zbl 1052.93061
It is well known that many problems of financial economics lead to the solutions of continuous-time, continuous-state stochastic control problems. Since to solve such problems exactly is very complicated, approximative methods have to be mostly employed. The diffusion continuous-time, continuous-state process is then replaced by a discrete-time, discrete-state Markov chain and, consequently, the original problem is (by this approach) replaced by a Markov chain control problem.
The aim of this publication is to compare various versions of the Markov chain approximation approach in the case of Merton’s simple consumption/investment problem for which an explicit solution is known. It follows from the paper that the above mentioned approximations are suitable. Of course, the behaviour of the approximation depends on the determination of an upper bound on the state space for the approximating Markov chain (namely the original process is unbounded).
Moreover, it follows from the paper that the Richardson extrapolation technique can be successively employed to improve the solution.
The aim of this publication is to compare various versions of the Markov chain approximation approach in the case of Merton’s simple consumption/investment problem for which an explicit solution is known. It follows from the paper that the above mentioned approximations are suitable. Of course, the behaviour of the approximation depends on the determination of an upper bound on the state space for the approximating Markov chain (namely the original process is unbounded).
Moreover, it follows from the paper that the Richardson extrapolation technique can be successively employed to improve the solution.
Reviewer: Vlasta Kaňková (Praha)
MSC:
| 93E20 | Optimal stochastic control |
| 91B42 | Consumer behavior, demand theory |
| 91G10 | Portfolio theory |
| 65C40 | Numerical analysis or methods applied to Markov chains |
| 60J22 | Computational methods in Markov chains |
Keywords:
Markov chain approximation; Merton’s consumption/investment problem; Richardson extrapolationReferences:
| [1] | Buchanan, J. L.; Turner, P. R., Numerical Methods and Analysis (1992), McGraw-Hill: McGraw-Hill New York |
| [2] | Bunch, D.; Johnson, H., A simple and numerically efficient valuation method for American puts using a modified Geske-Johnson approach, J. Finance, 47, 809-816 (1992) |
| [3] | Christiansen, E.; Petersen, H. G., Estimation of convergence orders in repeated Richardson extrapolation, BIT, 29, 48-59 (1989) · Zbl 0672.65006 |
| [4] | Duffie, D., Dynamic Asset Pricing Theory (1996), Princeton University Press: Princeton University Press Princeton, NJ |
| [5] | Fitzpatrick, B. G.; Fleming, W. H., Numerical methods for an optimal investment-consumption model, Math. Oper. Res., 16, 4, 823-841 (1991) · Zbl 0744.90003 |
| [6] | Fleming, W. H.; Soner, H. M., Controlled Markov Processes and Viscosity Solutions. Controlled Markov Processes and Viscosity Solutions, Applications of Mathematics, vol. 25 (1993), Springer: Springer New York · Zbl 0773.60070 |
| [7] | Geske, R.; Johnson, H. E., The American put valued analytically, J. Finance, 39, 5, 1511-1524 (1984) |
| [8] | Karatzas, I.; Lehoczky, J. P.; Sethi, S. P.; Shreve, S. E., Explicit solution of a general consumption/investment problem, Math. Oper. Res., 11, 2, 261-294 (1986) · Zbl 0622.90018 |
| [9] | Kushner, H. J., Probability Methods for Approximations in Stochastic Control and for Elliptic Equations. Probability Methods for Approximations in Stochastic Control and for Elliptic Equations, Mathematics in Science and Engineering, vol. 129 (1977), Academic Press: Academic Press New York · Zbl 0547.93076 |
| [10] | Kushner, H. J., Numerical methods for stochastic control problems in continuous time, SIAM J. Control Optim., 28, 5, 999-1048 (1990) · Zbl 0721.93087 |
| [11] | Kushner, H. J.; Dupuis, P. G., Numerical Methods for Stochastic Control Problems in Continuous Time. Numerical Methods for Stochastic Control Problems in Continuous Time, Applications of Mathematics, vol. 24 (1992), Springer: Springer New York · Zbl 0754.65068 |
| [12] | Merton, R. C., Lifetime portfolio selection under uncertainty: The continuous-time case, Rev. Econom. Statist., 51, 247-257 (1969), (Reprinted as Chapter 4 in Merton [15]) |
| [13] | Merton, R. C., Optimum consumption and portfolio rules in a continuous-time model, J. Econom. Theory, 3, 373-413 (1971), (Erratum: Merton [14]. Reprinted as Chapter 5 in Merton [15]) · Zbl 1011.91502 |
| [14] | Merton, R. C., Erratum, J. Econom. Theory, 6, 213-214 (1973) |
| [15] | Merton, R. C., Continuous-time finance (1992), Basil Blackwell Inc: Basil Blackwell Inc Padstow, UK |
| [16] | C. Munk, Numerical methods for continuous-time, continuous-state stochastic control problems, Publications from Department of Management 97/11, Odense University, 1997a; C. Munk, Numerical methods for continuous-time, continuous-state stochastic control problems, Publications from Department of Management 97/11, Odense University, 1997a |
| [17] | C. Munk, Optimal Consumption-Portfolio Policies and Contingent Claims Pricing and Hedging in Incomplete Markets, Ph.D. thesis, Odense University, DK-5230 Odense M, Denmark, 1997b; C. Munk, Optimal Consumption-Portfolio Policies and Contingent Claims Pricing and Hedging in Incomplete Markets, Ph.D. thesis, Odense University, DK-5230 Odense M, Denmark, 1997b |
| [18] | Rust, J., Numerical dynamic programming in economics, (Amman, H. M.; Kendrick, D. A.; Rust, J., Handbook of Computational Economics, vol. I (1996), North-Holland: North-Holland Amsterdam) · Zbl 1126.65316 |
| [19] | Tapiero, C. S.; Sulem, A., Computational aspects in applied stochastic control, Comput. Econom., 7, 109-146 (1994) · Zbl 0821.90026 |
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. In some cases that data have been complemented/enhanced by data from zbMATH Open. This attempts to reflect the references listed in the original paper as accurately as possible without claiming completeness or a perfect matching.
