Using dynamic programming with adaptive grid scheme for optimal control problems in economics. (English) Zbl 1202.49026
Summary: The study of the solutions of dynamic models with optimizing agents has often been limited by a lack of available analytical techniques to explicitly find the global solution paths. On the other hand, the application of numerical techniques such as dynamic programming to find the solution in interesting regions of the state was restricted by the use of fixed grid size techniques. Following L. Grüne [Numer. Math. 75, No. 3, 319–337 (1997; Zbl 0880.65045); Numer. Math. 99, No. 1, 85–112 (2004; Zbl 1074.65009)], in this paper an adaptive grid scheme is used for finding the global solutions of discrete time Hamilton-Jacobi-Bellman equations. Local error estimates are established and an adapting iteration for the discretization of the state space is developed. The advantage of the use of adaptive grid scheme is demonstrated by computing the solutions of one- and two-dimensional economic models which exhibit steep curvature, complicated dynamics due to multiple equilibria, thresholds (Skiba sets) separating domains of attraction and periodic solutions. We consider deterministic and stochastic model variants. The studied examples are from economic growth, investment theory, environmental and resource economics.
MSC:
| 49L20 | Dynamic programming in optimal control and differential games |
| 49N90 | Applications of optimal control and differential games |
| 91B55 | Economic dynamics |
| 49M25 | Discrete approximations in optimal control |
| 90C39 | Dynamic programming |
References:
| [1] | Azariadis, C.; Drazen, A., Thresholds externalities in economic development, Quarterly Journal of Economics, 105, 2, 501-526 (1990) |
| [2] | Bardi, M.; Capuzzo Dolcetta, I., Optimal Control and Viscosity Solutions of Hamilton-Jacobi-Bellman equations (1997), Birkhäuser: Birkhäuser Boston, MA · Zbl 0890.49011 |
| [4] | Benhabib, J.; Perli, R., Uniqueness and indeterminacyon the dynamics of endogenous growth, Journal of Economic Theory, 63, 1, 113-142 (1994) · Zbl 0803.90023 |
| [5] | Benhabib, J.; Perli, R.; Xie, D., Monopolistic competition, indeterminacy and growth, Ricerche Economiche, 48, 279-298 (1994) · Zbl 0826.90020 |
| [6] | Blanchard, O. J., Debt and current account deficit in Brazil, (Armella, P. A.; Dornbush, R.; Obstfield, M., Financial Policies and the World Capital Market: the Problem of Latin American Countries (1983), University of Chicago Press: University of Chicago Press Chicago, IL), 187-197 |
| [7] | Brock, W.; Mirman, L., Optimal economic growth and uncertainty: the discounted case, Journal of Economic Theory, 4, 479-513 (1972) |
| [9] | Burnside, C., Discrete state-space methods for the study of dynamic economies, (Marimon, R.; Scott, A., Computational Methods for the Study of Dynamic Economies (2001), Oxford University Press: Oxford University Press Oxford), 95-113 |
| [10] | Camilli, F.; Falcone, M., An approximation scheme for the optimal control of diffusion processes, RAIRO. Modélisation Math. Anal. Numér, 29, 97-122 (1995) · Zbl 0822.65044 |
| [11] | Candler, G. V., Finite-difference methods for continuous-time dynamic programming, (Marimon, R.; Scott, A., Computational Methods for the Study of Dynamic Economies (2001), Oxford University Press: Oxford University Press Oxford), 172-194 |
| [12] | Capuzzo Dolcetta, I., On a discrete approximation of the Hamilton-Jacobi equation of dynamic programming, Appl. Math. Optim, 10, 367-377 (1983) · Zbl 0582.49019 |
| [13] | Capuzzo Dolcetta, I.; Falcone, M., Discrete dynamic programming and viscosity solutions of the Bellman equation, Ann. Inst. Henri Poincaré, Anal. Non Linéaire, 6, supplement, 161-184 (1989) · Zbl 0674.49028 |
| [15] | Chow, C.-S.; Tsitsiklis, J. N., An optimal one-way multigrid algorithm for discrete-time stochastic control, IEEE Transactions on Automatic Control, 36, 898-914 (1991) · Zbl 0752.93078 |
| [16] | Daniel, J. W., Splines and efficiency in dynamic programming, Journal of Mathematical Analysis and Applications, 54, 402-407 (1976) · Zbl 0345.90041 |
| [19] | Falcone, M.; Ferretti, R., Convergence analysis for a class of high-order semi-Lagrangian advection schemes, SIAM Journal of Numerical Analysis, 35, 909-940 (1998) · Zbl 0914.65097 |
| [20] | Falcone, M.; Giorgi, T., An approximation scheme for evolutive Hamilton-Jacobi equations, (McEneaney, W. M.; Yin, G. G.; Zhang, Q., Stochastic Analysis, Control, Optimization and Applications (1999), Birkhäuser: Birkhäuser Boston, MA), 288-303 · Zbl 0931.65067 |
| [22] | Feichtinger, G.; Kort, P.; Hartl, R. F.; Wirl, F., The dynamics of a simple relative adjustment-cost framework, German Economic Review, 2, 3, 255-268 (2001) |
| [23] | González, R. L.V.; Sagastizábal, C. A., Un algorithme pour la résolution rapide d’équations discrètes de Hamilton-Jacobi-Bellman, C. R. Acad. Sci., Paris, Sér. I, 311, 45-50 (1990) · Zbl 0719.65053 |
| [24] | Grüne, L., An adaptive grid scheme for the discrete Hamilton-Jacobi-Bellman equation, Numerical Mathematics, 75, 3, 319-337 (1997) · Zbl 0880.65045 |
| [26] | Grüne, L.; Metscher, M.; Ohlberger, M., On numerical algorithm and interactive visualization for optimal control problems, Computing and Visualization in Science, 1, 4, 221-229 (1999) · Zbl 0970.65073 |
| [30] | Jermann, U. J., Asset pricing in production economies, Journal of Monetary Economics, 41, 257-275 (1998) |
| [31] | Johnson, S. A.; Stedinger, J. R.; Shoemaker, C. A.; Li, Y.; Tejada-Guibert, J. A., Numerical solution of continuous-state dynamic programs using linear and spline interpolation, Operations Research, 41, 484-500 (1993) · Zbl 0777.90074 |
| [33] | Judd, K. L., Numerical Methods in Economics (1998), MIT Press: MIT Press Cambridge, MA · Zbl 0941.00048 |
| [34] | Judd, K. L.; Guu, S.-M., Asymptotic methods for aggregate growth models, Journal of Economic Dynamics & Control, 21, 1025-1042 (1997) · Zbl 0901.90039 |
| [35] | Keane, M. P.; Wolpin, K. I., The solution and estimation of discrete choice dynamic programming models by simulation and interpolationMonte Carlo evidence, The Review of Economics & Statistics, 76, 648-672 (1994) |
| [36] | Ljungqvist, L.; Sargent, T. J., Recursive Macroeconomic Theory (2001), MIT Press: MIT Press Cambridge, MA |
| [37] | Marcet, A., Simulation analysis of stochastic dynamic models: applications to theory and estimation, (Sims, C. A., Advances in Econometrics, Sixth World Congress of the Econometric Society (1994), Cambridge University Press: Cambridge University Press Cambridge), 81-118 |
| [38] | Munos, R.; Moore, A., Variable resolution discretization in optimal control, Machine Learning, 49, 291-323 (2002) · Zbl 1005.68086 |
| [39] | Reiter, M., Solving higher-dimensional continuous-time stochastic control problems by value function regression, Journal of Economic Dynamics and Control, 23, 1329-1353 (1999) · Zbl 0949.93080 |
| [40] | Rust, J., Numerical Dynamic Programming in economics, (Amman, H. M.; Kendrick, D. A.; Rust, J., Handbook of Computational Economics (1996), Elsevier: Elsevier Amsterdam), 620-729 · Zbl 1126.65316 |
| [41] | Rust, J., Using randomization to break the curse of dimensionality, Econometrica, 65, 478-516 (1997) · Zbl 0872.90107 |
| [43] | Santos, M. S.; Vigo-Aguiar, J., Analysis of a numerical dynamic programming algorithm applied to economic models, Econometrica, 66, 2, 409-426 (1998) · Zbl 1010.90091 |
| [44] | Seeck, A., Iterative Lösungen der Hamilton-Jacobi-Bellman-Gleichung bei unendlichem Zeithorizont (1997), Diplomarbeit: Diplomarbeit Universität Kiel |
| [45] | Semmler, W.; Sieveking, M., On optimal exploitation of interacting resources, Journal of Economics, 59, 1, 23-49 (1994) · Zbl 0798.90018 |
| [46] | Sieveking, M.; Semmler, W., The Present value of resources with large discount rates, Applied Mathematics and Optimization, 35, 283-309 (1997) · Zbl 0881.49002 |
| [47] | Skiba, A. K., Optimal growth with a convex-concave a production function, Econometrica, 46, 3, 527-539 (1978) · Zbl 0383.90020 |
| [48] | Tauchen, G.; Hussey, R., Quadrature-based methods for obtaining approximate solutions to nonlinear asset-price models, Econometrica, 59, 371-396 (1991) · Zbl 0735.90012 |
| [49] | Taylor, J. B.; Uhlig, H., Solving non-linear stochastic growth modelsa comparison of alternative solution methods, Journal of Business and Economic Studies, 8, 1-18 (1990) |
| [51] | Trick, M. A.; Zin, S. E., Spline approximations to value functionsa linear programming approach, Macroeconomic Dynamics, 1, 255-277 (1997) · Zbl 0914.90209 |
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. In some cases that data have been complemented/enhanced by data from zbMATH Open. This attempts to reflect the references listed in the original paper as accurately as possible without claiming completeness or a perfect matching.
