Abstract
In this paper, a family of hybrid conjugate parameters with restart procedure is proposed. In which, we design a hybrid conjugate parameter by using two hybrid techniques, and set a restart procedure in its search direction according to the conjugate parameter. Under the usual assumptions and the weak Wolfe line search, we prove its sufficient descent property and global convergence. Finally, we choose a specific algorithm from this family and use an acceleration scheme to solve the large-scale unconstrained optimization problems and image restorations. These numerical results show that our algorithm is effective.
Similar content being viewed by others
Data availability
Data will be made available on request.
Notes
https://github.com/jhyin-optim/FHTTCGMs with applications
References
Hestenes, M.R., Stiefel, E.: Methods of conjugate gradients for solving linear systems. J. Res. Natl. Bur. Stand. 49(6), 409–436 (1952)
Fletcher, R., Reeves, C.M.: Function minimization by conjugate gradients. Comput. J. 7(2), 149–154 (1964)
Polak, E., Ribière, G.: Note surla convergence de directions conjugèes. Rev Fr Informat Rech Operationelle 3e Anneè 16(3), 35–43 (1969)
Polyak, B.T.: The conjugate gradient method in extremal problems. USSR Comput. Math. Math. Phys. 9(4), 94–112 (1969)
Dai, Y.H., Yuan, Y.X.: A nonlinear conjugate gradient method with a strong global convergence property. SIAM J. Optim. 10(1), 177–182 (1999)
Wei, Z.X., Yao, S.W., Liu, L.Y.: The convergence properties of some new conjugate gradient methods. Appl. Math. Comput. 183(2), 1341–1350 (2006)
Jiang, X.Z., Jian, J.B.: A sufficient descent Dai-Yuan type nonlinear conjugate gradient method for unconstrained optimization problems. Nonlinear Dyn. 72, 101–112 (2013)
Zhu, Z.B., Zhang, D.D., Wang, S.: Two modified DY conjugate gradient methods for unconstrained optimization problems. Appl. Math. Comput. 373(15), 125004 (2020)
Touati-Ahmed, D., Storey, C.: Efficient hybrid conjugate gradient techniques. J. Optim. Theory Appl. 64(2), 379–397 (1990)
Dai, Y.H., Yuan, Y.X.: An efficient hybrid conjugate gradient method for unconstrained optimization. Ann. Oper. Res. 103(1), 33–47 (2001)
Andrei, N.: Another hybrid conjugate gradient algorithm for unconstrained optimization. Numer. Algor. 47(2), 143–156 (2008)
Andrei, N.: Hybrid conjugate gradient algorithm for unconstrained optimization. J. Optim. Theory Appl. 141(2), 249–264 (2009)
Jian, J.B., Han, L., Jiang, X.Z.: A hybrid conjugate gradient method with descent property for unconstrained optimization. Appl. Math. Model. 39(3–4), 1281–1290 (2015)
Jiang, X.Z., Yang, H.H., Jian, J.B., et al.: Two families of hybrid conjugate gradient methods with restart procedures and their applications. Optim. Methods Softw. 38(5), 947–974 (2023)
Dai, Y.H., Liao, L.Z.: New conjugacy conditions and related nonlinear conjugate gradient methods. Appl. Math. Optim. 43, 87–101 (2001)
Yabe, H., Takano, M.: Global convergence properties of nonlinear conjugate gradient methods with modified secant condition. Comput. Optim. Appl. 28, 203–225 (2004)
Li, G., Tang, C., Wei, Z.: New conjugacy condition and related new conjugate gradient methods for unconstrained optimization. J. Comput. Appl. Math. 202, 523–539 (2007)
Babaie-Kafaki, S., Ghanbari, R.: The Dai-Liao nonlinear conjugate gradient method with optimal parameter choices. Eur. J. Oper. Res. 234, 625–630 (2014)
Shao, H., Guo, H., Wu, X.Y., et al.: Two families of self-adjusting spectral hybrid DL conjugate gradient methods and applications in image denoising. Appl. Math. Model. 118, 393–411 (2023)
Zhang, L., Zhou, W., Li, D.H.: Some descent three-term conjugate gradient methods and their global convergence. Optim. Methods Softw. 22(4), 697–711 (2007)
Jiang, X.Z., Liao, W., Yin, J.H., et al.: A new family of hybrid three-term conjugate gradient methods with applications in image restoration. Numer. Algor. 91, 161–191 (2022)
Jiang, X.Z., Yang, H.H., Yin, J.H., et al.: A three-term conjugate gradient algorithm with restart procedure to solve image restoration problems. J. Comput. Appl. Math. 424, 115020 (2023)
Babaie-Kafaki, S.: Two modified scaled nonlinear conjugate gradient methods. J. Comput. Appl. Math. 261, 172–182 (2014)
Andrei, N.: Scaled conjugate gradient algorithms for unconstrained optimization. Comput. Optim. Appl. 38(3), 401–416 (2007)
Jian, J.B., Chen, Q., Jiang, X.Z., et al.: A new spectral conjugate gradient method for large-scale unconstrained optimization. Optim. Methods Softw. 32(3), 503–515 (2017)
Crowder, H., Wolfe, P.: Linear convergence of the conjugate gradient method. IBM J. Res. Dev. 16(4), 431–433 (1972)
Powell, M.J.D.: Some convergence properties of the conjugate gradient method. Math. Program. 11(1), 42–49 (1976)
Andrei, N.: Open problems in nonlinear conjugate gradient algorithms for unconstrained optimization. Bull. Malays. Math. Sci. Soc. 34(2), 319–330 (2011)
Jiang, X.Z., Zhu, Y.H., Jian, J.B.: Two efficient nonlinear conjugate gradient methods with restart procedures and their applications in image restoration. Nonlinear Dyn. 111(6), 5469–5498 (2023)
Yin, J.H., Jian, J.B., Jiang, X.Z., et al.: A hybrid three-term conjugate gradient projection method for constrained nonlinear monotone equations with applications. Numer. Algor. 88(1), 389–418 (2022)
Liu, P.J., Wu, X.Y., Shao, H., et al.: Three adaptive hybrid derivative-free projection methods for constrained monotone nonlinear equations and their applications. Numer. Linear Algebra Appl. 30(2), 2471 (2022)
Liu, Y.F., Zhu, Z.B., Zhang, B.X.: Two sufficient descent three-term conjugate gradient methods for unconstrained optimization problems with applications in compressive sensing. J. Appl. Math. Comput. 68(3), 1787–1816 (2022)
Kou, C.X., Yang, H.: A mini-batch stochastic conjugate gradient algorithm with variance reduction. J. Glob. Optim. 87, 1–17 (2022)
Yang, Z.: Adaptive stochastic conjugate gradient for machine learning. Expert Syst. Appl. 206, 117719 (2022)
Shi, Z.J., Guo, J.: A new family of conjugate gradient methods. J. Comput. Appl. Math. 224(1), 444–457 (2009)
Nazareth, J.L.: Conjugate-gradient methods. In: Floudas, C., Pardalos, P. (eds.) Encyclopedia of Optimization. Kluwer Academic Publishers, Boston (1999)
Jiang, X.Z., Ye, X.M., Huang, Z.F., et al.: A family of hybrid conjugate gradient method with restart procedure for unconstrained optimizations and image restorations. Comput. Oper. Res. 159, 106–341 (2023)
Zoutendijk, G.: Nonlinear programming computational methods. In: Abadie, J. (ed.) Integer and Nonlinear Programming, pp. 37–86. North-Holland, Amsterdam (1970)
Andrei, N.: Acceleration of conjugate gradient algorithms for unconstrained optimization. Appl. Math. Comput. 213(2), 361–369 (2009)
Kou, C.X., Dai, Y.H.: A modified self-scaling memoryless Broyden-Fletcher-Goldfarb-Shanno method for unconstrained optimization. J. Optim. Theory Appl. 165(1), 209–224 (2015)
Dai, Y.H., Kou, C.X.: A nonlinear conjugate gradient algorithm with an optimal property and an improved Wolfe line search. SIAM J. Optim. 23(1), 296–320 (2013)
Andrei, N.: Hybrid conjugate gradient algorithm for unconstrained optimization. J. Optim. Theory Appl. 141(2), 249–264 (2009)
Gould, N.I.M., Orban, D., Toint, P.L.: CUTEr and SifDec: a constrained and unconstrained testing environment. ACM Trans. Math. Softw. 29(4), 373–394 (2003)
Moré, J.J., Garbow, B.S., Hillstrome, K.E.: Testing unconstrained optimization software. ACM Trans. Math. Softw. 7(1), 17–41 (1981)
Andrei, N.: An unconstrained optimization test functions collection. Adv. Model. Optim. 10(1), 147–161 (2008)
Dolan, E.D., Moré, J.J.: Benchmarking optimization software with performance profiles. Math. program. 91(2), 201–213 (2002)
Chan, R.H., Ho, C.W., Nikolova, M.: Salt-and-pepper noise removal by median-type noise detectors and detail-preserving regularization. IEEE Trans. Image Process. 14(10), 1479–1485 (2005)
Charbonnier, P., Blanc-Féraud, L., Aubert, G., et al.: Deterministic edge-preserving regularization in computed imaging. IEEE T. Image Process. 6(2), 298–311 (1997)
Cai, J.F., Chan, R., Morini, B.: Minimization of an edge-preserving regularization functional by conjugate gradient type methods. Image Processing Based on Partial Differential Equations. Springer, Berlin, Heidelberg. pp. 109–122 (2007)
Bovik, A.C.: Handbook of Image and Video Processing. Academic Press, San Diego (2010)
Hwang, H., Haddad, R.A.: Adaptive median filters: New algorithms and results. IEEE Trans. Image Process. 4(4), 499–502 (1995)
Acknowledgements
Project supported by the National Natural Science Foundation of China (Grant No. 12361063), Guangxi Science and Technology Program (Grant No. AD23023001), the Natural Science Foundation of Guangxi Province (Grant No.2016GXNSFAA380028).
Author information
Authors and Affiliations
Contributions
Xiaodi Wu: Validation, Algorithm design, Theoretical analysis, Writing-review. Xiaomin Ye: Theoretical analysis, Numerical experiments, Writing-review editing. Daolan Han: Numerical experiments.
Corresponding author
Ethics declarations
Conflict of interest
The authors declared no potential conflicts of interest with respect to the research, authorship, and publication of this article.
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Project supported by the National Natural Science Foundation of China (Grant No. 12361063), Guangxi Science and Technology Program (Grant No. AD23023001), the Natural Science Foundation of Guangxi Province (Grant No.2016GXNSFAA380028).
Rights and permissions
Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.
About this article
Cite this article
Wu, X., Ye, X. & Han, D. A family of accelerated hybrid conjugate gradient method for unconstrained optimization and image restoration. J. Appl. Math. Comput. 70, 2677–2699 (2024). https://doi.org/10.1007/s12190-024-02069-5
Received:
Revised:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s12190-024-02069-5
Keywords
- Unconstrained optimization
- Accelerated hybrid conjugate gradient method
- Restart procedure
- Global convergence
- Image restoration