A descent family of hybrid conjugate gradient methods with global convergence property for nonconvex functions. (English) Zbl 1524.90296
Summary: In this paper, we present a new hybrid conjugate gradient method for unconstrained optimization that possesses sufficient descent property independent of any line search. In our method, a convex combination of the Hestenes-Stiefel (HS) and the Fletcher-Reeves (FR) methods, is used as the conjugate parameter and the hybridization parameter is determined by minimizing the distance between the hybrid conjugate gradient direction and direction of the three-term HS method proposed by M. Li [Optim. Lett. 12, No. 8, 1911–1927 (2018; Zbl 1412.90165)]. Under some standard assumptions, the global convergence property on general functions is established. Numerical results on some test problems in the CUTEst library illustrate the efficiency and robustness of our proposed method in practice.
Keywords:
unconstrained optimization; conjugate gradient method; sufficient descent; least-squares; global convergenceCitations:
Zbl 1412.90165References:
| [1] | N. Andrei, Numerical comparison of conjugate gradient algorithms for unconstrained optimiza-tion, Stud. Inform. Control. 16 (2007) 333-352. |
| [2] | N. Andrei, Hybrid conjugate gradient algorithm for unconstrained optimization, J. Optim. Theory Appl. 141 (2009) 249-264. · Zbl 1168.90017 |
| [3] | N. Andrei, An adaptive conjugate gradient algorithm for large-scale unconstrained optimization, Comput. Appl. Math. 292 (2016) 83-91. · Zbl 1321.90124 |
| [4] | Z. Aminifard, S. Babaie-Kafaki, An optimal parameter choice for the Dai-Liao family of conjugate gradient methods by avoiding a direction of the maximum magnification by the search direction matrix, 4OR. 17 (2018) 1-14. |
| [5] | S. Babaie-Kafaki, M. Fatemi, N. Mahdavi-Amiri, Two effective hybrid conjugate gradient algo-rithms based on modified BFGS updates, Numer. Algor. 58 (2011) 315-331. · Zbl 1277.90149 |
| [6] | S. Babaie-Kafaki, R. Ghanbari, A hybridization of the Polak-Ribière-Polyak and Fletcher-Reeves conjugate gradient methods, Numer. Algor. 68(3) (2015) 481-495. · Zbl 1311.65066 |
| [7] | Y. Dai, L. Liao, New conjugate conditions and related nonlinear conjugate gradient methods, Appl. Math. Optim. 43 (2001) 87-101. · Zbl 0973.65050 |
| [8] | Y.H. Dai, J.Y. Han, G.H. Liu, D.F. Sun, H.X. Yin, Y.X. Yuan, Convergence properties of nonlinear conjugate gradient methods, SIAM J. Optim. 10(2) (1999) 348-358. · Zbl 0957.65062 |
| [9] | Y.H. Dai, Y. Yuan, A nonlinear conjugate gradient method with a strong global convergence prop-erty, SIAM J. Optim. 10 (1999) 177-182. · Zbl 0957.65061 |
| [10] | E. Dolan, J. Moré, Benchmarking optimization software with performance profiles, Math. 91 (2002) 201-213. · Zbl 1049.90004 |
| [11] | R. Fletcher, C.M. Reeves, Function minimization by conjugate gradients, Comput. J. 7(2) (1964) 149-154. · Zbl 0132.11701 |
| [12] | J.C. Gilbert, J. Nocedal, Global convergence properties of conjugate gradient methods for opti-mization, SIAM J. Optim. 2 (1992) 21-42. · Zbl 0767.90082 |
| [13] | N.I.M. Gould, D. Orban, P.L. Toint, CUTEst : a constrained and unconstrained testing environment with safe threads for mathematical optimization, Comput. Optim. Appl. 60(3) (2015) 545-557. · Zbl 1325.90004 |
| [14] | W.W. Hager, H.A. Zhang, New conjugate gradient method with guaranteed descent and an efficient line search, SIAM J. Optim. 16 (2005) 170-192. · Zbl 1093.90085 |
| [15] | M.R. Hestenes, E.L. Stiefel, Methods of conjugate gradients for solving linear systems, J. Res. Natl. Bur. Stand. (1952) 409-432. · Zbl 0048.09901 |
| [16] | J. Jian, Q. Chen, X. Jiang, Y. Zeng, J. Yin, A new spectral conjugate gradient method for large-scale unconstrained optimization, Optim. Methods Softw. 32 (3) (2017) 503515. · Zbl 1365.90201 |
| [17] | X. Jiang, J. Jian, Improved Fletcher-Reeves and Dai-Yuan conjugate gradient methods with the strong Wolfe line search, J. Comput. Appl. Math. 348 (2019) 525-534. · Zbl 1409.90092 |
| [18] | M. Li, A family of three-term nonlinear conjugate gradient methods close to the memoryless BFGS method, Optim. Lett. 12 (8) (2018) 1911-1927. · Zbl 1412.90165 |
| [19] | I.E. Livieris, V. Tampakas, P. Pintelas, A descent hybrid conjugate gradient method based on the memoryless BFGS update, Numer. Algor. 79 (4) (2018) 1169-1185. · Zbl 06986853 |
| [20] | M. Lotfi, S.M. Hosseini, An efficient Dai-Liao type conjugate gradient method by reformulating the CG parameter in the search direction equation, J. Comput. Appl. Math. (2020) 371:112708 · Zbl 1493.65111 |
| [21] | M. Lotfi, S.M. Hosseini, An efficient hybrid conjugate gradient method with sufficient descent prop-erty for unconstrained optimization, Optim. Methods Softw., 2021, https://doi.org/10.1080/ 10556788.2021.1977808. · Zbl 1511.65049 · doi:10.1080/10556788.2021.1977808 |
| [22] | J. Nocedal, S. Wright, Numerical Optimization, 2nd edition springer, New York, 2006. · Zbl 1104.65059 |
| [23] | E. Polak, G. Ribière, Note sur la convergence des mthodes de directions conjuguèes, Rev. Fr. Inform. Rech. Oper. 16 (1969) 35-43. · Zbl 0174.48001 |
| [24] | E.T. Polyak, The conjugate gradient method in extreme problems, USSR Comp. Math. Math. Phys. 9 (1969) 94-112. · Zbl 0229.49023 |
| [25] | M.J.D. Powell, Restart procedures for the conjugate gradient method, Math. Program. 12(2) (1977) 241-254. · Zbl 0396.90072 |
| [26] | G.L. Yuan, Modified nonlinear conjugate gradient methods with sufficient descent property for large-scale optimization problems, Optim. Lett. 3 (2009) 1121. |
| [27] | L. Zhang, W. Zhou, D. Li, Global convergence of a modified Fletcher-Reeves conjugate gradient method with Armijo-type line search, Numer. Math. 104 (2006) 561-572. · Zbl 1103.65074 |
| [28] | L. Zhang, W. Zhou, DH. Li, Some descent three-term conjugate gradient methods and their global convergence. Optim. Methods Softw. 2(4), (2007) 697-711. · Zbl 1220.90094 |
| [29] | Y. Zheng, B. Zheng, Two new Dai-Liao-type conjugate gradient methods for unconstrained opti-mization problems, J. Optim. Theory Appl. 175 (2017) 502-509.. · Zbl 1380.65108 |
| [30] | G. Zoutendijk, Nonlinear programming, computational methods, In: Abadie, J. (ed.) Integer and Nonlinear Programming, North-Holland, Amsterdam (1970) 37-86. · Zbl 0336.90057 |
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. In some cases that data have been complemented/enhanced by data from zbMATH Open. This attempts to reflect the references listed in the original paper as accurately as possible without claiming completeness or a perfect matching.
