Optimal scaling parameters for spectral conjugate gradient methods. (English) Zbl 1490.90304
Summary: To improve upon numerical stability of the spectral conjugate gradient methods, two adaptive scaling parameters are introduced. One parameter is obtained by minimizing an upper bound of the condition number of the matrix involved in producing the search direction and the other one is obtained by minimizing the Frobenius condition number of the matrix. The proposed methods are shown to be globally convergent, under appropriate conditions. A comparative testing of the proposed methods and some efficient spectral conjugate gradient methods shows the computational efficiency of the proposed methods.
MSC:
| 90C53 | Methods of quasi-Newton type |
| 49M37 | Numerical methods based on nonlinear programming |
| 65K05 | Numerical mathematical programming methods |
Keywords:
unconstrained optimization; spectral conjugate gradient method; condition number; global convergenceReferences:
| [1] | Birgin, E.; Martinez, JM, A spectral conjugate gradient method for unconstrained optimization, Appl Math Optim, 43, 117-128 (2001) · Zbl 0990.90134 · doi:10.1007/s00245-001-0003-0 |
| [2] | Andrei, N., Scaled conjugate gradient algorithms for unconstrained optimization, Comput Optim Appl, 38, 401-416 (2007) · Zbl 1168.90608 · doi:10.1007/s10589-007-9055-7 |
| [3] | Dehghani, R.; Mahdavi-Amiri, N., Scaled nonlinear conjugate gradient methods for nonlinear least squares problems, Numerical Algorithms, 79, 1-20 (2018) · Zbl 1398.65049 · doi:10.1007/s11075-017-0425-7 |
| [4] | Hager, WW; Zhang, H., Algorithm 851: CGDESCENT, a conjugate gradient method with guaranteed descent, ACM Trans Math Softw, 32, 113-137 (2006) · Zbl 1346.90816 · doi:10.1145/1132973.1132979 |
| [5] | Yu, GH; Guan, LT; Chen, WF, Spectral conjugate gradient methods with sufficient descent property for large-scale unconstrained optimization, Optim Methods Softw, 23, 275-293 (2008) · Zbl 1279.90166 · doi:10.1080/10556780701661344 |
| [6] | Zhang, Y.; Dan, B., An efficient adaptive scaling parameter for the spectral conjugate gradient method, Optim Lett, 10, 1, 119-136 (2016) · Zbl 1335.65056 · doi:10.1007/s11590-015-0865-8 |
| [7] | Sun, W.; Yuan, YX, Optimization theory and methods: nonlinear programming (2006), New York: Springer, New York · Zbl 1129.90002 |
| [8] | Nocedal, J.; Wright, SJ, Numerical optimization (2006), New York: Springer, New York · Zbl 1104.65059 |
| [9] | Fletcher R (2005) On the Barzilai-Borwein method. In: Qi L, Teo K, Yang X (eds) Optimization and Control Applications, Series: Applied Optimization, vol 96. Springer Berlin, pp 35-256 · Zbl 1118.90318 |
| [10] | Hager, WW; Zhang, H., A new conjugate gradient method with guaranteed descent and an efficient line search, SIAM J Optim, 16, 170-192 (2005) · Zbl 1093.90085 · doi:10.1137/030601880 |
| [11] | Dai, YH; Han, JY; Liu, GH; Sun, DF; Yin, HX; Yuan, YX, Convergence properties of nonlinear conjugate gradient methods, SIAM J Optim, 10, 348-358 (1999) · Zbl 0957.65061 · doi:10.1137/S1052623497318992 |
| [12] | Polak, E.; Ribiére, G., Note sur la convergence de directions conjugue, Francaise Informat Recherche Operationelle, 16, 35-43 (1969) · Zbl 0174.48001 |
| [13] | Polyak, BT, The conjugate gradient method in extreme problems, USSR Comput Math Math Phys, 9, 94-112 (1969) · Zbl 0229.49023 · doi:10.1016/0041-5553(69)90035-4 |
| [14] | Fletcher, R.; Revees, CM, Function minimization by conjugate gradients, Comput J, 7, 149-154 (1964) · Zbl 0132.11701 · doi:10.1093/comjnl/7.2.149 |
| [15] | Raydan, M., The Barzilai and Borwein gradient method for the large scale unconstrained minimization problem, SIAM J Optim, 7, 26-33 (1997) · Zbl 0898.90119 · doi:10.1137/S1052623494266365 |
| [16] | Shanno, DF; Phua, KH, Algorithm 500: minimization of unconstrained multivariate functions, ACM Trans Math Software, 2, 87-94 (1976) · Zbl 0319.65042 · doi:10.1145/355666.355673 |
| [17] | Gould, NIM; Orban, D.; Toint, PhL, CUTEr, a constrained and unconstrained testing environment, revisited, ACM Trans Math Softw, 29, 373-394 (2003) · Zbl 1068.90526 · doi:10.1145/962437.962439 |
| [18] | Dolan, ED; Moré, JJ, Benchmarking optimization software with performance profiles, Math Program, 91, 201-213 (2002) · Zbl 1049.90004 · doi:10.1007/s101070100263 |
| [19] | Hager, WW; Zhang, H., A survey of nonlinear conjugate gradient methods, Pac J Optim, 2, 35-58 (2006) · Zbl 1117.90048 |
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. In some cases that data have been complemented/enhanced by data from zbMATH Open. This attempts to reflect the references listed in the original paper as accurately as possible without claiming completeness or a perfect matching.
