A nonmonotone supermemory gradient algorithm for unconstrained optimization. (English) Zbl 1296.90116
Summary: This paper presents a nonmonotone supermemory gradient algorithm for unconstrained optimization problems. At each iteration, this proposed method sufficiently uses the previous multi-step iterative information and avoids the storage and computation of matrices associated with the Hessian of objective functions, thus it is suitable to solve large-scale optimization problems and can converge stably. Under some assumptions, the convergence properties of the proposed algorithm are analyzed. Numerical results are also reported to show the efficiency of this proposed method.
MSC:
| 90C30 | Nonlinear programming |
| 65K05 | Numerical mathematical programming methods |
| 49M37 | Numerical methods based on nonlinear programming |
Keywords:
unconstrained optimization; nonmonotone line search; supermemory gradient method; convergence analysisReferences:
| [1] | Sun, W.Y., Yuan, Y.X.: Optimization Theory and Methods: Nonlinear Programming. Springer Optimization and Its Applications, vol. 1. Springer, New York (2006) · Zbl 1129.90002 |
| [2] | Dai, Y.H., Yuan, Y.X.: Nonlinear Conjugate Gradient Methods. Shanghai Sci. Technol., Shanghai (2000) (in Chines) |
| [3] | Hager, W.W., Zhang, H.C.: A survey of nonlinear conjugate gradient methods. Pac. J. Optim. 2, 35-58 (2006) · Zbl 1117.90048 |
| [4] | Wei, Z.X., Li, G.Y., Qi, L.Q.: Global convergence of the Polak-Ribière-Polyak conjugate gradient methods with inexact line search for nonconvex unconstrained optimization problems. Math. Comput. 77, 2173-2193 (2008) · Zbl 1198.65091 · doi:10.1090/S0025-5718-08-02031-0 |
| [5] | Yuan, G.L.: Modified nonlinear conjugate gradient methods with sufficient descent property for large-scale optimization problems. Optim. Lett. 3, 11-21 (2009) · Zbl 1154.90623 · doi:10.1007/s11590-008-0086-5 |
| [6] | Zhang, L., Zhou, W.J., Li, D.H.: A descent modified Polak-Ribière-Polyak conjugate method and its global convergence. IMA J. Numer. Anal. 26, 629-649 (2006) · Zbl 1106.65056 · doi:10.1093/imanum/drl016 |
| [7] | Hager, W.W., Zhang, H.C.: A new conjugate gradient method with guaranteed descent and an efficient line search. SIAM J. Optim. 16, 170-192 (2005) · Zbl 1093.90085 · doi:10.1137/030601880 |
| [8] | Cheng, W.Y., Liu, Q.F.: Sufficient descent nonlinear conjugate gradient methods with conjugacy condition. Numer. Algorithms 53, 113-131 (2010) · Zbl 1185.65097 · doi:10.1007/s11075-009-9318-8 |
| [9] | Narushima, Y., Yabe, H., Ford, J.A.: A three-term conjugate gradient method with sufficient descent property for unconstrained optimization. SIAM J. Optim. 21, 212-230 (2011) · Zbl 1250.90087 · doi:10.1137/080743573 |
| [10] | Miele, A., Cantrell, J.W.: Study on a memory gradient method for the minimization of functions. J. Optim. Theory Appl. 3, 459-470 (1969) · Zbl 0165.22702 · doi:10.1007/BF00929359 |
| [11] | Cragg, E.E., Levy, A.V.: Study on a supermemory gradient method for the minimization of functions. J. Optim. Theory Appl. 4, 191-205 (1969) · Zbl 0172.19002 · doi:10.1007/BF00930579 |
| [12] | Narushima, Y., Yabe, H.: Global convergence of a memory gradient method for unconstrained optimization. Comput. Optim. Appl. 35, 325-346 (2006) · Zbl 1128.90059 · doi:10.1007/s10589-006-8719-z |
| [13] | Shi, Z.J., Shen, J.: A new supermemory gradient method with curve search rule. Appl. Math. Comput. 170, 1-16 (2005) · Zbl 1081.65058 · doi:10.1016/j.amc.2004.10.063 |
| [14] | Zheng, Y., Wan, Z.P.: A new variant of the memory gradient method for unconstrained optimization. Optim. Lett. 6, 1643-1655 (2012) · Zbl 1258.90070 · doi:10.1007/s11590-011-0355-6 |
| [15] | Shi, Z.J., Shen, J.: On memory gradient method with trust region for unconstrained optimization. Numer. Algorithms 41, 173-196 (2006) · Zbl 1098.65067 · doi:10.1007/s11075-005-9008-0 |
| [16] | Ou, Y.G., Wang, G.S.: A new supermemory gradient method for unconstrained optimization problems. Optim. Lett. 6, 975-992 (2012) · Zbl 1279.90165 · doi:10.1007/s11590-011-0328-9 |
| [17] | Ou, Y.G., Wang, G.S.: A hybrid ODE-based method for unconstrained optimization problems. Comput. Optim. Appl. 53, 249-270 (2012) · Zbl 1259.90139 · doi:10.1007/s10589-012-9455-1 |
| [18] | Grippo, L., Lampariello, F., Lucidi, S.: A nonmonotone line search technique for Newton’s method. SIAM J. Numer. Anal. 23, 707-716 (1986) · Zbl 0616.65067 · doi:10.1137/0723046 |
| [19] | Liu, G.H., Jing, L.Z., Han, L.X., Han, D.: A class of nonmonotone conjugate gradient method for unconstrained optimization. J. Optim. Theory Appl. 101, 127-140 (1999) · Zbl 0956.90047 · doi:10.1023/A:1021723128049 |
| [20] | Shi, Z.J., Wang, S.Q., Xu, Z.W.: The convergence of conjugate gradient method with nonmonotone line search. Appl. Math. Comput. 217, 1921-1932 (2010) · Zbl 1206.65166 · doi:10.1016/j.amc.2010.06.047 |
| [21] | Narushima, Y.: A nonmonotone memory gradient method for unconstrained optimization. J. Oper. Res. Soc. Jpn. 50, 31-45 (2007) · Zbl 1278.90387 |
| [22] | Raydan, M.: The Barzilai and Borwein gradient method for the large scale unconstrained minimization problem. SIAM J. Optim. 7, 26-33 (1997) · Zbl 0898.90119 · doi:10.1137/S1052623494266365 |
| [23] | Sun, W.Y., Zhou, Q.Y.: An unconstrained optimization method using nonmonotone second order Goldstein’s linesearch. Sci. China Ser. A, Math. 50, 1389-1400 (2007) · Zbl 1132.65057 · doi:10.1007/s11425-007-0072-x |
| [24] | Deng, N.Y., Xiao, Y., Zhou, F.J.: Nonmonotone trust region algorithm. J. Optim. Theory Appl. 76, 259-285 (1993) · Zbl 0797.90088 · doi:10.1007/BF00939608 |
| [25] | Mo, J.T., Zhang, K.C., Wei, Z.X.: A nonmonotone trust region method for unconstrained optimization. Appl. Math. Comput. 171, 371-384 (2005) · Zbl 1094.65059 · doi:10.1016/j.amc.2005.01.048 |
| [26] | Ou, Y.G., Zhou, Q.: A nonmonotonic trust region algorithm for a class of semi-infinite minimax programming. Appl. Math. Comput. 215(2), 474-480 (2009) · Zbl 1180.65080 · doi:10.1016/j.amc.2009.05.009 |
| [27] | Sun, W.Y.: Nonmonotone trust region method for solving optimization problems. Appl. Math. Comput. 156, 159-174 (2004) · Zbl 1059.65055 · doi:10.1016/j.amc.2003.07.008 |
| [28] | Ji, Y., Li, Y.J., Zhang, K.C., Zhang, X.L.: A new nonmonotone trust region method for conic model for solving unconstrained optimization. J. Comput. Appl. Math. 233, 1746-1754 (2010) · Zbl 1183.65067 · doi:10.1016/j.cam.2009.09.011 |
| [29] | Dai, Y.H.: On the nonmonotone line search. J. Optim. Theory Appl. 112, 315-330 (2002) · Zbl 1049.90087 · doi:10.1023/A:1013653923062 |
| [30] | Toint, Ph.L.: An assessment of nonmonotone line search techniques for unconstrained optimization. SIAM J. Sci. Stat. Comput. 17, 725-739 (1996) · Zbl 0849.90113 · doi:10.1137/S106482759427021X |
| [31] | Zhang, H.C., Hager, W.W.: A nonmonotone line search technique and its application to unconstrained optimization. SIAM J. Optim. 14, 1043-1056 (2004) · Zbl 1073.90024 · doi:10.1137/S1052623403428208 |
| [32] | Hu, S.L., Huang, Z.H., Lu, N.: A nonmonotone line search algorithm for unconstrained optimization. J. Sci. Comput. 42, 38-53 (2010) · Zbl 1203.90146 · doi:10.1007/s10915-009-9314-0 |
| [33] | Cui, Z.C., Wu, B.Y.: A new modified nonmonotone adaptive trust region method for unconstrained optimization. Comput. Optim. Appl. 53, 795-806 (2012) · Zbl 1264.90158 · doi:10.1007/s10589-012-9460-4 |
| [34] | Shi, Z.J., Shen, J.: Step-size estimation for unconstrained optimization methods. Comput. Appl. Math. 24, 399-416 (2005) · Zbl 1213.90232 · doi:10.1590/S1807-03022005000300005 |
| [35] | Andrei, N.: An unconstrained optimization test functions. Adv. Model. Optim. 10(1), 147-161 (2008) · Zbl 1161.90486 |
| [36] | Dolan, E.D., More, J.J.: Benchmarking optimization software with performance profiles. Math. Program., Ser. A 91, 201-213 (2002) · Zbl 1049.90004 · doi:10.1007/s101070100263 |
| [37] | Wen, Z.W., Yin, W.T., Liu, X., Zhang, Y.: Introduction to compressive sensing and sparse optimization. Oper. Res. Trans. 16, 49-64 (2012) (in Chinese) |
| [38] | Vasant, P.: Meta-Heuristics Optimization Algorithms in Engineering, Business, Economics, and Finance. IGI Global, Hershey (2013) |
| [39] | Vasant, P.M.: Handbook of Research on Novel Soft Computing Intelligent Algorithms: Theory and Practical Applications. IGI Global, Hershey (2014) |
| [40] | Kolda, T.G., Lewis, R.M., Torczon, V.: Optimization by direct search: new perspectives on some classical and modern methods. SIAM Rev. 45, 385-482 (2003) · Zbl 1059.90146 · doi:10.1137/S003614450242889 |
| [41] | Ganesan, T., Vasant, P., Elamvazuthi, I.: Hybrid neuro-genetic programming approach for optimizing operations in 3-D land seismic surveys. Math. Comput. Model. 54, 2913-2922 (2011) · Zbl 1235.90200 · doi:10.1016/j.mcm.2011.07.012 |
| [42] | Ganesan, T., Vasant, P., Elamvazuthi, I.: Hybrid PSO approach for solving non-convex optimization problems. Arch. Control Sci. 22, 5-23 (2012) |
| [43] | Vasant, P., Barsoum, N.: Hybrid genetic algorithms and line search method for industrial production planning with non-linear fitness function. Eng. Appl. Artif. Intell. 22, 767-777 (2009) · doi:10.1016/j.engappai.2009.03.010 |
| [44] | Vasant, P., Barsoum, N.: Hybrid pattern search and simulated annealing for fuzzy production planning problems. Comput. Math. Appl. 60, 1058-1067 (2010) · Zbl 1201.90211 · doi:10.1016/j.camwa.2010.03.063 |
| [45] | Vasant, P.: Hybrid simulated annealing and genetic algorithms for industrial production management problems. Int. J. Comput. Methods 7, 279-297 (2010) · Zbl 1267.91005 · doi:10.1142/S0219876210002209 |
| [46] | Vasant Hybrid, P.: LS-SA-PS methods for solving fuzzy non-linear programming problems. Math. Comput. Model. 57, 180-188 (2013) · Zbl 1305.90436 · doi:10.1016/j.mcm.2011.08.002 |
| [47] | Gong, B.: A new Fourier neural network based on Quasi-Newton method. ICIC Express Lett., Part B, Appl. 3, 355-360 (2012) |
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. In some cases that data have been complemented/enhanced by data from zbMATH Open. This attempts to reflect the references listed in the original paper as accurately as possible without claiming completeness or a perfect matching.
