Some three-term conjugate gradient methods with the new direction structure. (English) Zbl 1437.90163
Summary: In this paper, a three-term conjugate gradient method with the new direction structure is proposed for solving large-scale unconstrained optimization problems, which generates a sufficient descent direction in per-iteration by the aid of some inexact line search conditions. Under suitable assumptions, the proposed method is globally convergent for nonconvex smooth problems. We further generalize the new direction structure to other traditional methods and obtain some algorithms with the same structure as the proposed method. Preliminary numerical comparisons show that the proposed methods are effective and promising for the test problems.
MSC:
| 90C52 | Methods of reduced gradient type |
| 90C30 | Nonlinear programming |
| 90C06 | Large-scale problems in mathematical programming |
| 65K05 | Numerical mathematical programming methods |
Keywords:
unconstrained optimization; three-term conjugate gradient method; sufficient descent property; global convergence; numerical comparisonReferences:
| [1] | Fletcher, R.; Reeves, C. M., Function minimization by conjugate gradients, Comput. J., 7, 149-154 (1964) · Zbl 0132.11701 |
| [2] | Dai, Y. H.; Yuan, Y. X., Nonlinear conjugate gradient with a strong global convergence property, SIAM J. Optim., 10, 177-182 (1999) · Zbl 0957.65061 |
| [3] | Fletcher, R., Practical Methods of Optimization (1987), Wiley: Wiley New York · Zbl 0905.65002 |
| [4] | Polak, E.; Ribière, G., Note surla convergence des methodse de directions conjugees, Rev. Fr. Imform. Rech. Oper., 16, 35-43 (1969) · Zbl 0174.48001 |
| [5] | Polyak, B. T., The conjugate gradient method in extreme problems, USSR Comput. Math. Math. Phys., 9, 94-112 (1969) · Zbl 0229.49023 |
| [6] | Hestenes, M. R.; Stiefel, E. L., Methods of conjugate gradients for solving linear systems, J. Res. Natl. Bur. Stand., 5, 409-432 (1952) · Zbl 0048.09901 |
| [7] | Liu, Y.; Story, C., Efficient generalized conjugate gradient algorithms, part 1: theory, J. Optim. Theory Appl., 69, 129-137 (1992) · Zbl 0702.90077 |
| [8] | Wolfe, P., Convergence conditions for ascent methods, SIAM Rev., 11, 226-235 (1968) · Zbl 0177.20603 |
| [9] | Zhang, L.; Zhou, W. J.; Li, D. H., A descent modified Polak-Ribière-Polyak conjugate gradient method and its global convergence, IMA J. Numer. Anal., 26, 629-640 (2006) · Zbl 1106.65056 |
| [10] | Zhang, L.; Zhou, W. J.; Li, D. H., Some descent three-term conjugate gradient methods and theri global convergence, Optim. Methods Softw., 22, 697-711 (2007) · Zbl 1220.90094 |
| [11] | Zhang, J.; Xiao, Y.; Wei, Z., Nonlinear conjugate gradient methods with sufficient descent condition for large-scale unconstrained optimization, Math. Probl. Eng., 2009, Article 243290 pp. (2009) · Zbl 1184.65066 |
| [12] | Al-Bayati, A. Y.; Sharif, W. H., A new three-term conjugate gradient method for unconstrained optimization, Can. J. Sci. Eng. Math., 1, 108-124 (2010) |
| [13] | Andrei, N., On three-term conjugate gradient algorithms for unconstrained optimization, Appl. Math. Comput., 219, 6316-6327 (2013) · Zbl 1274.90372 |
| [14] | Andrei, N., A simple three-term conjugate gradient algorithm for unconstrained optimization, J. Comput. Appl. Math., 241, 19-29 (2013) · Zbl 1258.65056 |
| [15] | Andrei, N., A new three-term conjugate gradient algorithm for unconstrained optimization, Numer. Algorithms, 68, 305-321 (2015) · Zbl 1321.65092 |
| [16] | Liu, J. K.; Li, S. J., New three-term conjugate gradient method with guaranteed global convergence, Int. J. Comput. Math., 91, 1744-1754 (2014) · Zbl 1302.90212 |
| [17] | Liu, J. K.; Wu, X. S., New three-term conjugate gradient method for solving unconstrained optimization problems, ScienceAsia, 40, 295-300 (2014) |
| [18] | Liu, J. K.; Feng, Y. M.; Zou, L. M., Some three-term conjugate gradient methods with the inexact line search condition, Calcolo, 55, 16 (2018) · Zbl 1393.90116 |
| [19] | Yuan, G. L.; Zhang, M. J., A three-terms Polak-Ribière-Polyak conjugate gradient algorithm for large-scale nonlinear equations, J. Comput. Appl. Math., 286, 186-195 (2015) · Zbl 1316.90038 |
| [20] | Yuan, G. L.; Wei, Z. X.; Li, G. Y., A modified Polak-Ribière-Polyak conjugate gradient algorithm for nonsmooth convex programs, J. Comput. Appl. Math., 255, 86-96 (2014) · Zbl 1291.90315 |
| [21] | Deng, N. Y.; Li, Z., Global convergence of three terms conjugate gradient methods, Optim. Methods Softw., 4, 273-282 (1995) |
| [22] | Dai, Y. H.; Yuan, Y., A nonlinear conjugate gradient method with a strong global convergence property, SIAM J. Optim., 10, 177-182 (1999) · Zbl 0957.65061 |
| [23] | Nazareth, L., A conjugate direction algorithm without line search, J. Optim. Theory Appl., 23, 373-387 (1977) · Zbl 0348.65061 |
| [24] | Andrei, N., A modified Polak-Ribière-Polyak conjugate gradient algorithm for unconstrained optimization, Optimization, 60, 1457-1471 (2011) · Zbl 1233.90255 |
| [25] | Narushima, Y.; Yabe, H.; Ford, J. A., A three-term conjugate gradient method with sufficient descent property for unconstrained optimization, SIAM J. Optim., 21, 212-230 (2011) · Zbl 1250.90087 |
| [26] | Li, X. L.; Shi, J. J., A new conjugate gradient method based on quasi-Newton equation for unconstrained optimization, J. Comput. Appl. Math., 350, 372-379 (2019) · Zbl 1524.90295 |
| [27] | Dong, X. L.; liu, Z. X., An efficient adaptive three-term extension of the Hestenes-Stiefel conjugate gradient method, Optim. Methods Softw. (2019) · Zbl 1411.65081 |
| [28] | Nakayama, S.; Narushima, Y.; Yabe, H., Memoryless quasi-Newton methods based on spectral-scaling Broyden family for unconstrained optimization, J. Ind. Manag. Optim., 13, 1-21 (2018) |
| [29] | Yao, S. W.; Ning, L. S., An adaptive three-term conjugate gradient method based on self-scaling memoryless BFGS matrix, J. Comput. Appl. Math., 332, 72-85 (2018) · Zbl 1382.90116 |
| [30] | Baluch, B.; Salleh, Z.; Alhawarat, A., A New Modified Three-Term Hestenes-Stiefel Conjugate Gradient Method with Sufficient Descent Property and Its Global Convergence, J. Optim., 2018, Article 5057096 pp. (2018) · Zbl 1460.90205 |
| [31] | Dong, X. L.; Han, D. R., An accelerated three-term conjugate gradient method with sufficient descent condition and conjugacy condition, J. Optim. Theory Appl., 179, 944-961 (2018) · Zbl 1402.90176 |
| [32] | Yuan, G. L.; Li, T. T.; Hu, W. J., A conjugate gradient algorithm for large-scale nonlinear equations and image restoration problems, Appl. Numer. Math., 147, 129-141 (2020) · Zbl 1433.90165 |
| [33] | Liu, J. K.; Xu, J. L.; Zhang, L. Q., Partially symmetrical derivative-free Liu-Storey projection method for convex constrained equations, Int. J. Comput. Math., 96, 1787-1798 (2019) · Zbl 1499.90221 |
| [34] | Zoutendijk, G., Nonlinear programming, computational methods, (Abadie, J., Integer and Nonlinear Programming (1970), North-Holland: North-Holland Amsterdam), 37-86 · Zbl 0336.90057 |
| [35] | Gilbert, J.; Nocedal, J., Global convergence properties of conjugate gradient methods for optimization, SIAM J. Optim., 2, 21-42 (1992) · Zbl 0767.90082 |
| [36] | Dai, Y. H.; Liao, L. Z., New conjugacy conditions and related nonlinear conjugate gradient methods, Appl. Math. Optim., 43, 87-101 (2001) · Zbl 0973.65050 |
| [37] | Hager, W. W.; Zhang, H., A new conjugate gradient method with guaranteed descent and an efficient line search, SIAM J. Optim., 16, 170-192 (2005) · Zbl 1093.90085 |
| [38] | Bongartz, I.; Conn, A. R.; Gould, N. I.M.; Toint, P. L., CUTE: constrained and unconstrained testing environments, ACM Trans. Math. Softw., 21, 123-160 (1995) · Zbl 0886.65058 |
| [39] | Andrei, N., An unconstrained optimization test functions collection, Adv. Model. Optim., 10, 147-161 (2008) · Zbl 1161.90486 |
| [40] | Dolan, E. D.; Moré, J. J., Benchmarking optimization software with performance profiles, Math. Program., 91, 201-213 (2002) · Zbl 1049.90004 |
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. In some cases that data have been complemented/enhanced by data from zbMATH Open. This attempts to reflect the references listed in the original paper as accurately as possible without claiming completeness or a perfect matching.
