×
Dong, Xiao-Liang; Liu, Ze-Xian; Liu, Hong-Wei; Li, Xiang-Li

An efficient adaptive three-term extension of the Hestenes-Stiefel conjugate gradient method. (English) Zbl 1411.65081

Optim. Methods Softw. 34, No. 3, 546-559 (2019).
Summary: A new three-term Hestenes-Stiefel-type conjugate gradient method is proposed in which the search direction can satisfy the sufficient descent condition as well as an adaptive conjugacy condition. It is notable that search directions of the method are dynamically adjusted between that of the Newton method and the 3HS\(+\) method, accelerating the convergence or reducing the condition number of iteration matrix. Under mild conditions, we show that the proposed method converges globally for general objective functions. Numerical experiments indicate that the method is practically promising.

Cited in 2 Documents

MSC:

65K05 Numerical mathematical programming methods
90C53 Methods of quasi-Newton type

Keywords:

unconstrained optimization; conjugate gradient method; conjugacy condition; sufficient descent condition; global convergence; numerical comparison
Cite Review PDF
Full Text: DOI

References:

[1] Al-Baali, M., Descent property and global convergence of the Fletcher-Reeves method with inexact line search, IMA J. Numer. Anal., 5, 1, 121-124 (1985) · Zbl 0578.65063 · doi:10.1093/imanum/5.1.121
[2] Al-Baali, M.; Narushima, Y.; Yabe, H., A family of three-term conjugate gradient methods with sufficient descent property for unconstrained optimization, Comput. Optim. Appl., 194, 2, 469-74 (2015) · Zbl 1315.90051
[3] Andrei, N., A simple three-term conjugate gradient algorithm for unconstrained optimization, J. Comput. Appl. Math., 241, 19-29 (2013) · Zbl 1258.65056 · doi:10.1016/j.cam.2012.10.002
[4] Andrei, N., On three-term conjugate gradient algorithms for unconstrained optimization, Appl. Math. Comput., 219, 11, 6316-6327 (2013) · Zbl 1274.90372
[5] Andrei, N., A new three-term conjugate gradient algorithm for unconstrained optimization, J. Comput. Appl. Math., 68, 2, 305-321 (2015) · Zbl 1321.65092
[6] Babaie-Kafaki, S.; Ghanbari, R., Two modified three-term conjugate gradient method with sufficient descent property, Optim. Lett., 8, 8, 2285-2297 (2014) · Zbl 1309.90097 · doi:10.1007/s11590-014-0736-8
[7] Cheng, W., A two-term PRP-based descent method, Numer. Funct. Anal. Opt., 28, 1217-1230 (2007) · Zbl 1138.90028 · doi:10.1080/01630560701749524
[8] Cheng, W.; Li, D., An active set modified Polak-Ribiére-Polyak method for large-scale nonlinear bound constrained optimization, J. Optim. Theory Appl., 155, 3, 1084-1094 (2012) · Zbl 1276.90067 · doi:10.1007/s10957-012-0091-9
[9] Conn, A. R.; Bongartz, I.; Gould, N., Constrained and unconstrained testing environment, ACM Trans. Math. Softw., 50, 124, 123-160 (1970) · Zbl 0886.65058
[10] Dai, Y. H., Conjugate gradient methods with Armijo-type line searches, Acta Math. Appl. Sin. Engl. Ser., 18, 1, 123-130 (2002) · Zbl 1114.90479 · doi:10.1007/s102550200010
[11] Dai, Y. H.; Kou, C.-X., A nonlinear conjugate gradient algorithm with an optimal property and an improved wolfe line search, SIAM J. Optim., 23, 1, 296-320 (2013) · Zbl 1266.49065 · doi:10.1137/100813026
[12] Dai, Y. H.; Liao, L., New conjugacy conditions and related nonlinear conjugate gradient methods, Appl. Math. Optim., 43, 1, 87-101 (2001) · Zbl 0973.65050 · doi:10.1007/s002450010019
[13] Dai, Y. H.; Yuan, Y., A nonlinear conjugate gradient method with a strong global convergence property, SIAM J. Optim., 10, 1, 177-182 (1999) · Zbl 0957.65061 · doi:10.1137/S1052623497318992
[14] Dai, Z.; Chen, X.; Wen, F., A modified Perry’s conjugate gradient method-based derivative-free method for solving large-scale nonlinear monotone equations, Appl. Math. Comput., 270, 7, 378-386 (2015) · Zbl 1410.90248
[15] Dai, Z.; Li, D.; Wen, F., Worse-case conditional value-at-risk for asymmetrically distributed asset scenarios returns, J. Comput. Anal. Appl., 20, 2, 237-251 (2016) · Zbl 1333.91060
[16] Deng, S.; Wan, Z., A three-term conjugate gradient algorithm for large-scale unconstrained optimization problems, Appl. Numer. Math., 92, 70-81 (2015) · Zbl 1321.65096 · doi:10.1016/j.apnum.2015.01.008
[17] Dolan, E. D.; Jorge, J. J., Benchmarking optimization software with performance profiles, Math. Program., 91, 2, 201-213 (2001) · Zbl 1049.90004 · doi:10.1007/s101070100263
[18] Dong, X., Comment on ‘a new three-term conjugate gradient method for unconstrained problem’, Numer. Algorithms, 72, 1, 173-179 (2015) · Zbl 1346.90765 · doi:10.1007/s11075-015-0039-x
[19] Dong, X.; Liu, H.; He, Y., A self-adjusting conjugate gradient method with sufficient descent condition and conjugacy condition, J. Optim. Theory Appl., 165, 1, 225-241 (2015) · Zbl 1322.90094 · doi:10.1007/s10957-014-0601-z
[20] Dong, X.; Liu, H. W.; He, Y. B.; Yang, X. M., A modified Hestenes-Stiefel conjugate gradient method with sufficient descent condition and conjugacy condition, J. Comput. Appl. Math., 281, 239-249 (2015) · Zbl 1309.65074 · doi:10.1016/j.cam.2014.11.058
[21] Dong, X.; Liu, H.; He, Y.; Babaie-Kafaki, S.; Ghanbari, R., A new three-term conjugate gradient method with descent direction for unconstrained optimization, Math. Model. Anal., 21, 3, 399-411 (2016) · Zbl 1488.49059 · doi:10.3846/13926292.2016.1176965
[22] Dong, X.; Han, D.-R.; Ghanbari, R.; Li, X.-L.; Dai, Z.-F., Some new three-term Hestenes-Stiefel conjugate gradient methods with affine combination, Optimization, 66, 5, 759-776 (2017) · Zbl 1375.90281 · doi:10.1080/02331934.2017.1295242
[23] Fletcher, R.; Reeves, C., Function minimization by conjugate gradients, Comput. J., 7, 2, 149-154 (1964) · Zbl 0132.11701 · doi:10.1093/comjnl/7.2.149
[24] Gilbert, J. C.; Nocedal, J., Global convergence properties of conjugate gradient methods for optimization, SIAM J. Optim., 2, 1, 21-42 (1992) · Zbl 0767.90082 · doi:10.1137/0802003
[25] Hager, W. W.; Zhang, H., A new conjugate gradient method with guaranteed descent and an efficient line search, SIAM J. Optim., 16, 1, 170-192 (2005) · Zbl 1093.90085 · doi:10.1137/030601880
[26] Hager, W. W.; Zhang, H., A survey of nonlinear conjugate gradient methods, Pac. J. Optim., 2, 1, 35-58 (2006) · Zbl 1117.90048
[27] Hager, W. W.; Zhang, H., The limited memory conjugate gradient method, SIAM J. Optim., 23, 4, 2150-2168 (2013) · Zbl 1298.90129 · doi:10.1137/120898097
[28] Hestenes, M. R.; Stiefel, E., Methods of conjugate gradients for solving linear systems, J. Res. Natl. Bur. Stand., 49, 6, 409-436 (1952) · Zbl 0048.09901 · doi:10.6028/jres.049.044
[29] Li, X.; Dong, X.; Zhang, W., A class of modified FR conjugate gradient method and applications to non-negative matrix factorization, Comput. Math. Appl., 73, 2, 270-276 (2017) · Zbl 1368.65097 · doi:10.1016/j.camwa.2016.11.017
[30] Narushima, Y.; Yabe, H.; Ford, J. A., A three-term conjugate gradient method with sufficient descent property for unconstrained optimization, SIAM J. Optim., 21, 1, 212-230 (2011) · Zbl 1250.90087 · doi:10.1137/080743573
[31] Sugiki, K.; Narushima, Y.; Yabe, H., Globally convergent three-term conjugate gradient methods that use secant conditions and generate descent search directions for unconstrained optimization, J. Optim. Theory Appl., 153, 3, 733-757 (2012) · Zbl 1262.90170 · doi:10.1007/s10957-011-9960-x
[32] Sun, L.; Fang, L.; He, G., An active set strategy based on the multiplier function or the gradient, Appl. Math., 55, 4, 291-304 (2010) · Zbl 1224.90176 · doi:10.1007/s10492-010-0022-8
[33] Sun, L.; He, G.; Wang, Y.; Zhou, C., An accurate active set newton algorithm for large scale bound constrained optimization, Appl. Math., 56, 3, 297-314 (2011) · Zbl 1224.90177 · doi:10.1007/s10492-011-0018-z
[34] Wen, F.; He, Z.; Dai, Z., Characteristics of investors’risk preference for stock markets, Econ. Comput. Econ. Cyb., 3, 48, 235-254 (2014)
[35] Wolfe, P., Convergence conditions for ascent methods, SIAM Rev., 11, 2, 226-235 (1969) · Zbl 0177.20603 · doi:10.1137/1011036
[36] Yu, J.; Li, M.; Wang, Y., A decomposition method for large-scale box constrained optimization, Appl. Math. Comput., 231, 12, 9-15 (2014) · Zbl 1410.90207
[37] Zhang, L.; Zhou, W.; Li, D., Global convergence of a modified Fletcher-Reeves conjugate gradient method with armijo-type line search, Numer. Math., 104, 561-572 (2006) · Zbl 1103.65074 · doi:10.1007/s00211-006-0028-z
[38] Zhang, L.; Zhou, W.; Li, D.-H., A descent modified Polak-Ribiére-Polyak conjugate gradient method and its global convergence, IMA J. Numer. Anal., 26, 4, 629-640 (2006) · Zbl 1106.65056 · doi:10.1093/imanum/drl016
[39] Zhang, L.; Zhou, W.; Li, D., Some descent three-term conjugate gradient methods and their global convergence, Optim. Methods Softw., 22, 4, 697-711 (2007) · Zbl 1220.90094 · doi:10.1080/10556780701223293
[40] Zheng, F.; Han, C.; Wang, Y., Parallel SSLE algorithm for large scale constrained optimization, Appl. Math., 217, 12, 5377-5384 (2011) · Zbl 1210.90164
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. In some cases that data have been complemented/enhanced by data from zbMATH Open. This attempts to reflect the references listed in the original paper as accurately as possible without claiming completeness or a perfect matching.