A new accelerated conjugate gradient method for large-scale unconstrained optimization. (English) Zbl 1499.90216
Summary: In this paper, we present a new conjugate gradient method using an acceleration scheme for solving large-scale unconstrained optimization. The generated search direction satisfies both the sufficient descent condition and the Dai-Liao conjugacy condition independent of line search. Moreover, the value of the parameter contains more useful information without adding more computational cost and storage requirements, which can improve the numerical performance. Under proper assumptions, the global convergence result of the proposed method with a Wolfe line search is established. Numerical experiments show that the given method is competitive for unconstrained optimization problems, with a maximum dimension of 100,000.
MSC:
| 90C30 | Nonlinear programming |
| 65K05 | Numerical mathematical programming methods |
| 90C52 | Methods of reduced gradient type |
| 90C53 | Methods of quasi-Newton type |
| 90C06 | Large-scale problems in mathematical programming |
Keywords:
conjugate gradient; descent condition; Dai-Liao conjugacy condition; global convergence; large-scale unconstrained optimizationSoftware:
CGOPTReferences:
| [1] | Andrei, N., A simple three-term conjugate gradient algorithm for unconstrained optimization, J. Comput. Appl. Math., 241, 19-29 (2013) · Zbl 1258.65056 · doi:10.1016/j.cam.2012.10.002 |
| [2] | Andrei, N., On three-term conjugate gradient algorithms for unconstrained optimization, Appl. Math. Comput., 219, 6316-6327 (2013) · Zbl 1274.90372 |
| [3] | Andrei, N., A new three-trem conjugate gradient algorithm for unconstrained optimization, Numer. Algorithms, 68, 305-321 (2015) · Zbl 1321.65092 · doi:10.1007/s11075-014-9845-9 |
| [4] | Babaie-Kafaki, S.; Ghanbari, R., A descent family of Dai-Liao conjugate gradient methods, Optim. Methods Softw., 29, 583-591 (2014) · Zbl 1285.90063 · doi:10.1080/10556788.2013.833199 |
| [5] | Babaie-Kafaki, S.; Ghanbari, R., The Dai-Liao nonlinear conjugate gradient method with optimal parameter choices, Eur. J. Oper. Res., 234, 625-630 (2014) · Zbl 1304.90216 · doi:10.1016/j.ejor.2013.11.012 |
| [6] | Babaie-Kafaki, S.; Ghanbari, R., Two optimal Dai-Liao conjugate gradient methods, Optimization, 64, 2277-2287 (2014) · Zbl 1386.65158 · doi:10.1080/02331934.2014.938072 |
| [7] | Babaie-Kafaki, S.; Ghanbari, R.; Mahdavi-Amiri, N., Two new conjugate gradient methods based on modified secant equations, J. Comput. Appl. Math., 234, 1374-1386 (2010) · Zbl 1202.65071 · doi:10.1016/j.cam.2010.01.052 |
| [8] | Dai, Y. H.; Liao, L. Z., New conjugacy conditions and related nonlinear conjugate gradient methods, Appl. Math. Optim., 43, 87-101 (2001) · Zbl 0973.65050 · doi:10.1007/s002450010019 |
| [9] | Dai, Y. H.; Yuan, Y. X., An efficient hybrid conjugate gradient method for unconstrained optimization, Ann. Oper. Res., 103, 33-47 (2001) · Zbl 1007.90065 · doi:10.1023/A:1012930416777 |
| [10] | Dai, Z. F., Comments on a new class of nonlinear conjugate gradient coefficients with global convergence properties, Appl. Math. Comput., 276, 297-300 (2016) · Zbl 1410.65234 |
| [11] | Dai, Z. F.; Chen, X. H.; Wen, F. H., A modified Perry’s conjugate gradient method-based derivative-free method for solving large-scale nonlinear monotone equations, Appl. Math. Comput., 270, 378-386 (2015) · Zbl 1410.90248 |
| [12] | Dai, Z. F.; Chen, X. H.; Wen, F. H., Comments on “A hybrid conjugate gradient method based on a quadratic relaxation of the Dai-Yuan hybrid conjugate gradient parameter”, Optimization, 64, 1173-1175 (2015) · Zbl 1310.65071 · doi:10.1080/02331934.2013.840783 |
| [13] | Dai, Z. F.; Wen, F. H., Comments on another hybrid conjugate gradient algorithm for unconstrained optimization by Andrei, Numer. Algorithms, 69, 337-341 (2015) · Zbl 1483.65095 · doi:10.1007/s11075-014-9899-8 |
| [14] | Deng, S. H.; Wan, Z., A three-term conjugate gradient algorithm for large-scale unconstrained optimization problems, Appl. Numer. Math., 92, 70-81 (2015) · Zbl 1321.65096 · doi:10.1016/j.apnum.2015.01.008 |
| [15] | Dolan, E.; Moré, J. J., Benchmarking optimization software with performance profiles, Math. Program., 91, 201-213 (2002) · Zbl 1049.90004 · doi:10.1007/s101070100263 |
| [16] | Flether, R.; Reeves, C. M., Function minimization by conjugate gradients, Comput. J., 7, 149-154 (1964) · Zbl 0132.11701 · doi:10.1093/comjnl/7.2.149 |
| [17] | Ford, J. A.; Narushima, Y.; Yabe, H., Multi-step nonlinear conjugate gradient methods for unconstrained minimization, Comput. Optim. Appl., 40, 191-216 (2008) · Zbl 1181.90221 · doi:10.1007/s10589-007-9087-z |
| [18] | Hager, W. W.; Zhang, H. C., A new conjugate gradient method with guaranteed descent and an efficient line search, SIAM J. Optim., 16, 170-192 (2005) · Zbl 1093.90085 · doi:10.1137/030601880 |
| [19] | Hestenes, M. R.; Stiefel, E. L., Methods of conjugate gradients for solving linear systems, J. Res. Natl. Bur. Stand., 49, 409-436 (1952) · Zbl 0048.09901 · doi:10.6028/jres.049.044 |
| [20] | Huang, C. X.; Yang, Z. C.; Yi, T. S.; Zou, X. F., On the basins of attraction for a class of delay differential equations with non-monotone bistable nonlinearities, J. Differ. Equ., 256, 2101-2114 (2014) · Zbl 1297.34084 · doi:10.1016/j.jde.2013.12.015 |
| [21] | Kou, C. X., An improved nonlinear conjugate gradient method with an optimal property, Sci. China Math., 57, 635-648 (2014) · Zbl 1303.90101 · doi:10.1007/s11425-013-4682-1 |
| [22] | Li, D. H.; Fukushima, M., A modified BFGS method and its global convergence in nonconvex minimization, J. Comput. Appl. Math., 129, 15-35 (2001) · Zbl 0984.65055 · doi:10.1016/S0377-0427(00)00540-9 |
| [23] | Liu, C. Y.; Gong, Z. H.; Teo, K. L.; Sun, J.; Caccetta, L., Robust multi-objective optimal switching control arising in \(1,31, 3\)-propanediol microbial fed-batch process, Nonlinear Anal. Hybrid Syst., 25, 1-20 (2017) · Zbl 1378.49043 · doi:10.1016/j.nahs.2017.01.006 |
| [24] | Liu, S.; Chen, Y. P.; Huang, Y. Q.; Zhou, J., An efficient two grid method for miscible displacement problem approximated by mixed finite element methods, Comput. Math. Appl., 77, 752-764 (2019) · Zbl 1442.65269 · doi:10.1016/j.camwa.2018.10.013 |
| [25] | Livieris, I. E.; Pintelas, P., A descent Dai-Liao conjugate gradient method based on a modified secant equation and its global convergence, ISRN Comput. Math., 2012 (2012) · Zbl 1245.65067 · doi:10.5402/2012/435495 |
| [26] | Narushima, Y.; Yabe, H., Conjugate gradient methods based on secant conditions that generate descent search directions for unconstrained optimization, J. Comput. Appl. Math., 236, 4303-4317 (2012) · Zbl 1258.65059 · doi:10.1016/j.cam.2012.01.036 |
| [27] | Perry, A., Technical note—a modified conjugate gradient algorithm, Oper. Res., 26, 1073-1078 (1978) · Zbl 0419.90074 · doi:10.1287/opre.26.6.1073 |
| [28] | Polak, E., Ribiére, G.: Note sur la convergence des méthodes de directions conjuguées. Rev. Fr. Inform. Rech. Oper., 3e Année 16, 35-43 (1969) · Zbl 0174.48001 |
| [29] | Polyak, B. T., The conjugate gradient method in extreme problems, USSR Comput. Math. Math. Phys., 9, 94-112 (1969) · Zbl 0229.49023 · doi:10.1016/0041-5553(69)90035-4 |
| [30] | Sugiki, K.; Narushima, Y.; Yabe, H., Globally convergent three-term conjugate gradient methods that use secant conditions and generate descent search directions for unconstrained optimization, J. Optim. Theory Appl., 153, 733-757 (2012) · Zbl 1262.90170 · doi:10.1007/s10957-011-9960-x |
| [31] | Wang, J. F.; Chen, X. Y.; Huang, L. H., The number and stability of limit cycles for planar piecewise linear systems of node-saddle type, J. Math. Anal. Appl., 469, 405-427 (2019) · Zbl 1429.34037 · doi:10.1016/j.jmaa.2018.09.024 |
| [32] | Wang, J. F.; Huang, C. X.; Huang, L. H., Discontinuity-induced limit cycles in a general planar piecewise linear system of saddle-focus type, Nonlinear Anal. Hybrid Syst., 22, 162-178 (2019) · Zbl 1431.34020 · doi:10.1016/j.nahs.2019.03.004 |
| [33] | Wolfe, P., Convergence conditions for ascent methods, SIAM Rev., 11, 226-235 (1969) · Zbl 0177.20603 · doi:10.1137/1011036 |
| [34] | Wolfe, P., Convergence conditions for ascent methods, II: some corrections, SIAM Rev., 13, 185-188 (1971) · Zbl 0216.26901 · doi:10.1137/1013035 |
| [35] | Yabe, H.; Takano, M., Global convergence properties of nonlinear conjugate gradient methods with modified secant condition, Comput. Optim. Appl., 28, 203-225 (2004) · Zbl 1056.90130 · doi:10.1023/B:COAP.0000026885.81997.88 |
| [36] | Yang, Y. T.; Chen, Y. T.; Lu, Y. L., A subspace conjugate gradient algorithm for large-scale unconstrained optimization, Numer. Algorithms, 76, 813-828 (2017) · Zbl 1379.65038 · doi:10.1007/s11075-017-0284-2 |
| [37] | Yao, S. W.; Ning, L. S., An adaptive three-term conjugate gradient method based on self-scaling memoryless BFGS matrix, J. Comput. Appl. Math., 322, 72-85 (2018) · Zbl 1382.90116 · doi:10.1016/j.cam.2017.10.013 |
| [38] | Yuan, J. L.; Zhang, Y. D.; Ye, J. X.; Xie, J.; Teo, K. L.; Zhu, X.; Feng, E. M.; Yin, H. C.; Xiu, Z. L., Robust parameter identification using parallel global optimization for a batch nonlinear enzyme-catalytic time-delayed process presenting metabolic discontinuities, Appl. Math. Model., 46, 554-571 (2017) · Zbl 1443.92042 · doi:10.1016/j.apm.2017.01.079 |
| [39] | Zhang, L.; Jian, S. Y., Further studies on the Wei-Yao-Liu nonlinear conjugate gradient method, Appl. Math. Comput., 219, 7616-7621 (2013) · Zbl 1295.90104 |
| [40] | Zhou, W. J., A short note on the global convergence of the unmodified PRP method, Optim. Lett., 7, 1367-1372 (2013) · Zbl 1276.90072 · doi:10.1007/s11590-012-0511-7 |
| [41] | Zhou, W. J., On the convergence of the modified Levenberg-Marquardt method with a nonmonotone second order Armijo type line search, J. Comput. Appl. Math., 239, 152-161 (2013) · Zbl 1258.65050 · doi:10.1016/j.cam.2012.09.025 |
| [42] | Zhou, W. J.; Chen, X. L., On the convergence of a modified regularized Newton method for convex optimization with singular solutions, J. Comput. Appl. Math., 239, 179-188 (2013) · Zbl 1255.65117 · doi:10.1016/j.cam.2012.09.030 |
| [43] | Zhou, W. J.; Shen, D. M., An inexact PRP conjugate gradient method for symmetric nonlinear equations, Numer. Funct. Anal. Optim., 35, 370-388 (2014) · Zbl 1320.90087 · doi:10.1080/01630563.2013.871290 |
| [44] | Zhou, W. J.; Zhang, L., A nonlinear conjugate gradient method based on the MBFGS secant condition, Optim. Methods Softw., 21, 707-714 (2006) · Zbl 1112.90096 · doi:10.1080/10556780500137041 |
| [45] | Zoutendijk, G.; Abadie, J. (ed.), Nonlinear programming, computational method, 37-86 (1970), Amsterdam · Zbl 0228.90034 |
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. In some cases that data have been complemented/enhanced by data from zbMATH Open. This attempts to reflect the references listed in the original paper as accurately as possible without claiming completeness or a perfect matching.
