A modified secant equation quasi-Newton method for unconstrained optimization. (English) Zbl 1509.65050
Summary: One of the most prominent iterative approaches for solving unconstrained optimization problems is the quasi-Newton method. Their fast convergence and exceptional precision distinguish the quasi-Newton methods. We propose a modified secant relation based on a quadratic model to better approximate the objective function’s second curvature. We then provide a new BFGS method for resolving unconstrained optimization problems based on this modified secant relationship. The proposed method uses both gradient and function values, while the usual Secant relation uses only gradient values. Under appropriate conditions, we show that the proposed method is globally convergent without needing any convexity assumption on the objective function. Comparative results show the computational efficiency of the proposed method from the iteration count and function/gradient evaluations perspective.
MSC:
| 65K05 | Numerical mathematical programming methods |
| 90C26 | Nonconvex programming, global optimization |
| 90C53 | Methods of quasi-Newton type |
Keywords:
unconstrained optimization; nonlinear optimization; quasi-Newton methods; modified secant methodsReferences:
| [1] | Abu Arqub, O.; Rashaideh, H., The RKHS method for numerical treatment for integrodifferential algebraic systems of temporal two-point BVPs, Neural Comput. Appl., 30, 2595-2606 (2018) · doi:10.1007/s00521-017-2845-7 |
| [2] | Abu Arqub, O., Computational algorithm for solving singular Fredholm time-fractional partial integrodifferential equations with error estimates, J. Appl. Math. Comput., 59, 227-243 (2019) · Zbl 1434.65314 · doi:10.1007/s12190-018-1176-x |
| [3] | Al-Baali, M., New property and global convergence of the Fletcher-Reeves method with inexact line searches, IMA J. Numer. Anal, 5, 122-124 (1985) · Zbl 0578.65063 · doi:10.1093/imanum/5.1.121 |
| [4] | Al-Baali, M.; Grandinetti, G., On the behavior of damped quasi-newton methods for unconstrained optimization, Iran. J. Oper. Res., 3, 1-10 (2012) |
| [5] | Broyden, CG, The convergence of a class of double-rank minimization algorithms—Part 2: the new algorithm J, . Inst. Math. Appl., 6, 222-231 (1970) · Zbl 0207.17401 · doi:10.1093/imamat/6.3.222 |
| [6] | Dai, Y.; Yuan, J.; Yuan, Y., Modified two-point step size Gradient methods for unconstrained optimization, Comput. Optim. Appl., 22, 3, 103-109 (2002) · Zbl 1008.90056 · doi:10.1023/A:1014838419611 |
| [7] | Dehghani, R.; Hosseini, MM; Bidabadi, N., The modified quasi-Newton methods for solving unconstrained optimization problems, Int. J. Numer. Model., 32, 11-22 (2019) · Zbl 1424.90257 · doi:10.1002/jnm.2459 |
| [8] | Fletcher, R.: An overview of unconstrained optimization, vol. 434 Mathematical and Physical Sciences (1994) · Zbl 0828.90123 |
| [9] | Ford, JA; Moghrabi, IA, Alternative parameter choices for multi-step quasi-newton methods, Optim. Methods Softw., 2, 357-370 (1993) · doi:10.1080/10556789308805550 |
| [10] | Ford, JA; Moghrabi, IAR, Multi-step quasi-Newton methods for optimization, J. Comput. Appl. Math., 50, 305-323 (1994) · Zbl 0807.65062 · doi:10.1016/0377-0427(94)90309-3 |
| [11] | Ford, JA; Moghrabi, IAR, Using function-values in multi-step quasi-Newton methods, J. Comput. Appl. Math., 66, 201-211 (1996) · Zbl 0856.65073 · doi:10.1016/0377-0427(95)00178-6 |
| [12] | Gould, NIM; Orban, D.; Toint, PL, CUTEr and SifDec: a con- strained and unconstrained testing environment, ACM Trans. Math. Softw., 29, 4, 373-394 (2003) · Zbl 1068.90526 · doi:10.1145/962437.962439 |
| [13] | Hassan, B.; Ghada, M., A new quasi-Newton equation on the gradient methods for optimization minimization problem, Indones. J. Electr. Eng. Comput. Sci., 19, 737-744 (2020) |
| [14] | Hassan, B., A new quasi-Newton updating formula based on the new quasi-Newton equation, Numer. Algebr. Control Optim., 4, 63-72 (2019) |
| [15] | Hassan, B.; Mohammed, T., A new variants of quasi-Newton equation based on the quadratic function for unconstrained optimization, Indones. J. Electr. Eng. Comput. Sci., 19, 2, 701-708 (2020) |
| [16] | Li, D.; Fukushima, M., A modified BFGS method and its global convergence in nonconvex minimization, J. Comput. Appl. Math., 129, 15-35 (2001) · Zbl 0984.65055 · doi:10.1016/S0377-0427(00)00540-9 |
| [17] | Mascarenhas, WF, The BFGS method with exact line searches fails for non-convex objective functions, Math. Program., 99, 49-61 (2004) · Zbl 1082.90108 · doi:10.1007/s10107-003-0421-7 |
| [18] | Moghrabi, IAR, Implicit extra-update multi-step quasi-newton methods, Int. J. Oper. Res., 28, 3, 69-81 (2017) · Zbl 1362.90358 |
| [19] | Moghrabi, IAR, New two-step conjugate gradient method for unconstrained optimization, Int. J. Appl. Math., 33, 853-866 (2020) · doi:10.12732/ijam.v33i5.8 |
| [20] | Moghrabi, IAR, A non-Secant quasi-Newton method for unconstrained nonlinear optimization, Cogent Eng., 9, 20-36 (2022) · doi:10.1080/23311916.2021.2018929 |
| [21] | Moghrabi, IAR, Metric-based parameterizations for multi-step unconstrained optimization, Comput. Optim. Appl., 19, 337-345 (2001) · Zbl 0988.90032 · doi:10.1023/A:1011242809975 |
| [22] | Momani, S.; Abu Arqub, B.; Maayah, B., Piecewise optimal fractional reproducing kernel solution and convergence analysis for the Atangana-Baleanu-Caputo model of the Lienard’s equation, Fractals, 28, 1-13 (2020) · Zbl 07468589 · doi:10.1142/S0218348X20400071 |
| [23] | Momani, S., Maayah, B., Abu Arqub, O.: The reproducing kernel algorithm for numerical solution of van der pol damping model in view of the Atangana-Baleanu fractional approach. Fractals 28, 2040010-1-2040010-12 (2020) · Zbl 07468592 |
| [24] | Moré, JJ; Garbow, BS; Hillstrom, KE, Testing unconstrained optimization software, ACM Trans. Math. Softw., 7, 17-41 (1981) · Zbl 0454.65049 · doi:10.1145/355934.355936 |
| [25] | Nakayama, S.; Narushima, Y.; Yabe, H., A memoryless symmetric rank-one method with sufficient descent property for unconstrained optimization, J. Oper. Res. Soc. Jpn., 61, 53-70 (2018) · Zbl 1391.90493 |
| [26] | Powell, M.J.D.: Some global convergence properties of a variable metric algorithm for minimization without exact line searches. In: SIAM-AMS Proceedings, Lemke, (eds.) SIAM, pp. 53-72 (1976) · Zbl 0338.65038 |
| [27] | Wei, Z.; Li, G.; Qi, L., The superlinear convergence of a modified BFGS- type method for unconstrained optimization, Comput. Optim. Appl., 29, 315-332 (2004) · Zbl 1070.90089 · doi:10.1023/B:COAP.0000044184.25410.39 |
| [28] | Wei, Z.; Li, G.; Qi, L., New quasi-Newton methods for unconstrained optimization problems, Appl. Math. Comput., 175, 1156-1188 (2006) · Zbl 1100.65054 |
| [29] | Wolfe, P., Convergence conditions for ascent methods, SIAM Rev., 11, 226-235 (1969) · Zbl 0177.20603 · doi:10.1137/1011036 |
| [30] | Xiao, YH; Wei, ZX; Zhang, L., A modified BFGS method without line searches for nonconvex unconstrained optimization, Adv. Theor. Appl. Math., 1, 149-162 (2006) · Zbl 1173.90585 |
| [31] | Yuan, G.; Sheng, Z.; Wang, B.; Hu, W.; Li, C., The global convergence of a modified BFGS method for nonconvex functions, J. Comput. Appl. Math., 327, 274-294 (2018) · Zbl 1370.90203 · doi:10.1016/j.cam.2017.05.030 |
| [32] | Yuan, G.; Wang, Z.; Li, P., A modifed Broyden family algorithm with global convergence under a weak Wolfe-Powell line search for unconstrained nonconvex problems, Calcolo, 57, 35-47 (2020) · Zbl 1467.90045 · doi:10.1007/s10092-020-00383-5 |
| [33] | Yuan, G.; Wei, Z.; Wu, Y., Modified limited memory BFGS method with non-monotone line search for unconstrained optimization, J. Korean Math. Soc., 47, 767-788 (2010) · Zbl 1193.65105 · doi:10.4134/JKMS.2010.47.4.767 |
| [34] | Yuan, G.; Wei, Z., Convergence analysis of a modified BFGS method on convex minimizations, Comput. Optim. Appl., 47, 237-255 (2010) · Zbl 1228.90077 · doi:10.1007/s10589-008-9219-0 |
| [35] | Yuan, G.; Wei, Z.; Lu, X., Global convergence of BFGS and PRP methods under a modified weak Wolfe-Powell line search, Appl. Math. Model., 47, 811-825 (2017) · Zbl 1446.65031 · doi:10.1016/j.apm.2017.02.008 |
| [36] | Zhang, J.; Deng, N.; Chen, L., Quasi-Newton equation and related methods for unconstrained optimization, JOTA, 102, 147-167 (1999) · Zbl 0991.90135 · doi:10.1023/A:1021898630001 |
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. In some cases that data have been complemented/enhanced by data from zbMATH Open. This attempts to reflect the references listed in the original paper as accurately as possible without claiming completeness or a perfect matching.
