A conjugate directions approach to improve the limited-memory BFGS method. (English) Zbl 1288.90095
Summary: Simple modifications of the limited-memory BFGS method (L-BFGS) for large scale unconstrained optimization are considered, which consist in corrections (derived from the idea of conjugate directions) of the used difference vectors, utilizing information from the preceding iteration. For quadratic objective functions, the improvement of convergence is the best one in some sense and all stored difference vectors are conjugate for unit stepsizes. Global convergence of the algorithm is established for convex sufficiently smooth functions. Numerical experiments indicate that the new method often improves the L-BFGS method significantly.
Keywords:
unconstrained minimization; variable metric methods; limited-memory methods; the BFGS update; conjugate directions; numerical resultsReferences:
| [1] | Andrei, N., An unconstrained optimization test functions collection, Advanced Modeling and Optimization, 10, 147-161 (2008) · Zbl 1161.90486 |
| [2] | Bongartz, I.; Conn, A. R.; Gould, N.; Toint, P. L., CUTE constrained and unconstrained testing environment, ACM Transactions on Mathematical Software, 21, 123-160 (1995) · Zbl 0886.65058 |
| [3] | Dolan, E. D.; Moré, J. J., Benchmarking optimization software with performance profiles, Mathematical Programming, 91, 201-213 (2002) · Zbl 1049.90004 |
| [4] | Fletcher, R., Practical Methods of Optimization (1987), John Wiley & Sons: John Wiley & Sons Chichester · Zbl 0905.65002 |
| [5] | Griewank, A.; Toint, Ph. L., Local convergence analysis for partitioned quasi-Newton updates, Numerische Mathematik, 39, 429-448 (1982) · Zbl 0505.65018 |
| [6] | Liu, D. C.; Nocedal, J., On the limited memory BFGS method for large scale optimization, Mathematical Programming, 45, 503-528 (1989) · Zbl 0696.90048 |
| [7] | L. Lukšan, C. Matonoha, J. Vlček, Sparse test problems for unconstrained optimization, Report V-1064, Prague, ICS AS CR, 2010.; L. Lukšan, C. Matonoha, J. Vlček, Sparse test problems for unconstrained optimization, Report V-1064, Prague, ICS AS CR, 2010. |
| [8] | L. Lukšan, C. Matonoha, J. Vlček, Modified CUTE problems for sparse unconstrained optimization, Report V-1081, Prague, ICS AS CR, 2010.; L. Lukšan, C. Matonoha, J. Vlček, Modified CUTE problems for sparse unconstrained optimization, Report V-1081, Prague, ICS AS CR, 2010. |
| [9] | Lukšan, L.; Spedicato, E., Variable metric methods for unconstrained optimization and nonlinear least squares, Journal of Computational and Applied Mathematics, 124, 61-95 (2000) · Zbl 0985.65066 |
| [10] | Moughrabi, I. A., New implicit multistep quasi-Newton methods, Numerical Analysis and Applications, 2, 154-164 (2009) |
| [11] | Nocedal, J., Updating quasi-Newton matrices with limited storage, Mathematics of Computation, 35, 773-782 (1980) · Zbl 0464.65037 |
| [12] | Nocedal, J.; Wright, S. J., Numerical Optimization (1999), Springer Verlag: Springer Verlag New York · Zbl 0930.65067 |
| [13] | Powell, M. J.D., The convergence of variable metric methods for nonlinearly constrained optimization calculations, (Mangasarian, O. L.; Meyer, R. R.; Robinson, S. M., Nonlinear Programming 3 (1978), Academic Press: Academic Press New York), 27-63 · Zbl 0464.65042 |
| [14] | J. Vlček, L. Lukšan, Generalizations of the limited-memory BFGS method based on quasi-product form of update, Report V-1060, Prague, ICS AS CR, 2009.; J. Vlček, L. Lukšan, Generalizations of the limited-memory BFGS method based on quasi-product form of update, Report V-1060, Prague, ICS AS CR, 2009. |
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. In some cases that data have been complemented/enhanced by data from zbMATH Open. This attempts to reflect the references listed in the original paper as accurately as possible without claiming completeness or a perfect matching.
