×

Parallel two-phase methods for global optimization on GPU. (English) Zbl 1540.65170

Summary: Developing general global optimization algorithms is a difficult task, specially for functions with a huge number of local minima in high dimensions. Stochastic metaheuristic algorithms can provide the only alternative for the solution of such problems since they are aimed at guaranteeing global optimality. However, the main drawback of these algorithms is that they require a large number of function evaluations in order to skip/discard local optima, thus exhibiting a low convergence order and, as a result, a high computational cost. Furthermore, the situation can become even worse with the increase of dimension. Usually the number of local minima highly increases, as well as the computational cost of the function evaluation, thus increasing the difficulty for covering the whole search space. On the other hand, deterministic local optimization methods exhibit faster convergence rates, requiring a lower number of functions evaluations and therefore involving a lower computational cost, although they can get stuck into local minima. A way to obtain faster global optimization algorithms is to mix local and global methods in order to benefit from higher convergence rates of local ones, while retaining the global approximation properties. Another way to speedup global optimization algorithms comes from the use of efficient parallel hardware architectures. Nowadays, a good alternative is to take advantage of graphics processing units (GPUs), which are massively parallel processors and have become quite accessible cheap alternative for high performance computing. In this work a parallel implementation on GPUs of some hybrid two-phase optimization methods, that combine the metaheuristic Simulated Annealing algorithm for finding a global minimum, with different local optimization methods, namely a conjugate gradient algorithm and a version of Nelder-Mead method, is presented. The performance of parallelized versions of the above hybrid methods are analyzed for a set of well known test problems. Results show that GPUs represent an efficient alternative for the parallel implementation of two-phase global optimization methods.

MSC:

65K05 Numerical mathematical programming methods
65Y05 Parallel numerical computation
68M14 Distributed systems
90C26 Nonconvex programming, global optimization
Full Text: DOI

References:

[1] Aarts, E.; van Laarhoven, P., Statistical cooling: A general approach to combinatorial optimization problems, Philips J. Res., 40, 193-226 (1985)
[2] Addis, B., Global Optimization using Local Searches (2004), Universitá degli studi di Firenze, (Ph.d. thesis)
[3] Addis, B.; Locatelli, M.; Schoen, F., Local optima smoothing for global optimizations, Optim. Methods Softw., 20, 417-437 (2005) · Zbl 1134.90035
[4] Armard, P., Modification of the Wolfe line search rules to satisfy the descent condition in the Polak-Ribière-Polyak conjugate gradient method, J. Optim. Theory Appl., 2, 287-305 (2007) · Zbl 1145.90070
[5] Breiman, L.; Cutler, A., A deterministic algorithm for global optimization, Math. Progr., 58, 179-199 (1993) · Zbl 0807.90103
[6] Broyden, C. G., The convergence of a class of double rank minimization algorithms: 2. The new algorithm, J. Inst. Math. Appl., 6, 222-231 (1970) · Zbl 0207.17401
[7] Byrd, R. H.; Lu, P.; Nocedal, J.; Zhu, C., A limited memory algorithm for bound constrained optimization, SIAM J. Sci. Comput., 16, 1190-1208 (1995) · Zbl 0836.65080
[8] Correia, A.; Matias, J.; Mestre, P.; Serdio, C., Derivative-free optimization and filter methods to solve nonlinear constrained problems, Int. J. Comput. Math., 86, 10-11 (2009) · Zbl 1177.65085
[9] F. Delbos, L. Dumas, E. Echague, Global optimization based on sparse grid surrogate models for black-box expensive functions, in: Proceedings of ECMOR XIV - 14th European Conference on the Mathematics of Oil Recovery, 2014.; F. Delbos, L. Dumas, E. Echague, Global optimization based on sparse grid surrogate models for black-box expensive functions, in: Proceedings of ECMOR XIV - 14th European Conference on the Mathematics of Oil Recovery, 2014.
[10] Dennis, J.; Torczon, V., Direct search methods on parallel machines, SIAM J. Optim., 1, 448-474 (1991) · Zbl 0754.90051
[11] Fan, S. S.; Zahara, E., A hybrid simplex search and particle swarm for unconstrained optimization, European J. Oper. Res., 181, 2, 527-548 (2007) · Zbl 1121.90116
[12] Ferreiro, A. M.; Rodríguez, J. A.G.; López-Salas, J.; Vázquez, C., An efficient implementation of parallel simulated annealing algorithm in GPUs, J. Global Optim., 57, 863-890 (2013) · Zbl 1286.90115
[13] Fletcher, R., A new approach to variable metric algorithms, Comput. J., 13, 317-322 (1970) · Zbl 0207.17402
[14] Fletcher, R., Practical methods of optimization, (Vol. 1: Unconstrained Optimization (1980), Wiley) · Zbl 0439.93001
[15] Fletcher, R.; Reeves, C. M., Function minimization by conjugate gradients, Comput. J., 7, 149-154 (1964) · Zbl 0132.11701
[16] Gao, F.; Han, L., Implementing the nelder-mead simplex algorithm with adaptive parameters, Comput. Optim. Appl., 51, 1, 259-277 (2012) · Zbl 1245.90121
[17] Gilbert, J.; Nocedal, L., Global convergence properties of conjugate gradient methods for optimization, SIAM J. Optim., 2, 21-42 (1992) · Zbl 0767.90082
[18] J. Gilbert, L. Nocedal, R. Waltz, CG+, 1992, http://users.iems.northwestern.edu/ nocedal/CG+.html; J. Gilbert, L. Nocedal, R. Waltz, CG+, 1992, http://users.iems.northwestern.edu/ nocedal/CG+.html
[19] Goffe, W. L., Simann: A global optimization algorithm using simulated annealing, Stud. Nonlinear Dyn. Econom., 1, 3, 1-9 (1996) · Zbl 1078.91519
[20] Goldfarb, D., A family of variable metric methods derived by variational means, Math. Comp., 24, 23-26 (1970) · Zbl 0196.18002
[21] Griewank, A., General descent for global optimization, J. Optim. Theory Appl., 34, 11-39 (1981) · Zbl 0431.49036
[22] Hager, W. W.; Zhang, H., A survey of nonlinear conjugate gradient methods, Pac. J. Optim., 2, 35-58 (2006) · Zbl 1117.90048
[23] Hedar, A. R.; Fukushima, M., Hybrid simulated annealing and direct search method for nonolinear unconstrained global optimization, Optim. Methods Softw., 17, 891-912 (2002) · Zbl 1065.90081
[24] Hedar, A.; Fukushima, M., Simplex coding genetic algorithm for the global optimization of nonlinear functions, (Multi-Objective Programming and Goal Programming. Multi-Objective Programming and Goal Programming, Advances in Soft Computing Series, vol. 21 (2003), Springer), 135-140 · Zbl 1165.90594
[25] Hedar, A.-R.; Fukushima, M., Heuristic pattern Search and its hybridization with simulated annealing for nonlinear global optimization, Optim. Methods Softw., 19, 291-308 (2004) · Zbl 1059.90149
[26] Hooke, R.; Jeeves, T., Direct search solution of numerical and statistical problems, J. Assoc. Comput. Mach., 8, 212-229 (1961) · Zbl 0111.12501
[27] Ingber, L., Pswarm: a hybrid solver for linearly constrained global derivative-free optimization, Math. Comput. Modelling, 18, 29-57 (1993) · Zbl 0819.90080
[28] Ingber, L., Adaptive simulated annealing (ASA): Lessons learned, Control Cybernet., 25, 33-54 (1996) · Zbl 0860.93035
[29] Kelley, C., Detection and remediation of stagnation in the nelder-mead algorithm using a sufficient decrease condition, SIAM J. Optim., 10, 43-53 (1999) · Zbl 0962.65048
[30] Kirkpatrick, S.; Gelatt, C. D.; Vecchi, M. P., Optimization by simulated annealing, Science, 4598, 671-680 (1983) · Zbl 1225.90162
[31] Lagarias, J.; Reeds, J.; Wright, M.; Wright, P., Convergence properties of the Nelder-Mead simplex methods in low dimensions, SIAM J. Optim., 9, 112-147 (1998) · Zbl 1005.90056
[32] Li, Z.; Scheraga, H. A., Monte Carlo minimization approach to the multiple-minima problem in protein folding, Proc. Natl. Acad. Sci. USA, 84, 6611-6615 (1987)
[33] Liu, D. C.; Nocedal, J., On the limited memory method for large scale optimization, Math. Progr. B, 45, 503-528 (1989) · Zbl 0696.90048
[34] Liuzzi, G.; Lucidi, S.; Piccialli, V.; Sotgiu, A., A magnetic resonance device designed via global optimization techniques, Math. Program., 101, 2, 339-364 (2004) · Zbl 1058.92030
[35] Locatelli, M., On the multilevel structure of global optimization problems, Comput. Optim. Appl., 30, 5-22 (2005) · Zbl 1130.90035
[36] Locatelli, M.; Schoen, F., (Global Optimization: Theory, Algorithms, and Applications. Global Optimization: Theory, Algorithms, and Applications, MOS-SIAM Series on Optimization (2013), SIAM) · Zbl 1286.90003
[37] McKinnon, K., Convergence of Nelder-Mead simplex method to a nonstationary point, SIAM J. Optim., 9, 148-158 (1998) · Zbl 0962.65049
[38] Metropolis, N.; Rosenbluth, A. W.; Rosenbluth, M. N.; Teller, A. H.; Teller, E., Equation of state calculations by fast computing machines, J. Chem. Phys., 21, 1087-1092 (1953) · Zbl 1431.65006
[39] J.J. Moré, D.J. Thuente, Minpack-2, 1993, http://ftp.mcs.anl.gov/pub/MINPACK-2/; J.J. Moré, D.J. Thuente, Minpack-2, 1993, http://ftp.mcs.anl.gov/pub/MINPACK-2/
[40] Moré, J. J.; Thuente, D. J., Line search algorithms with guaranteed sufficient decrease, ACM Trans. Math. Software, 20, 286-307 (1994) · Zbl 0888.65072
[41] Nelder, J. A.; Mead, R., A simplex method for function minimization, Comput. J., 7, 308-313 (1965) · Zbl 0229.65053
[42] J. Nocedal, L-BFGS, 1990, http://users.iems.northwestern.edu/ nocedal/lbfgs.html; J. Nocedal, L-BFGS, 1990, http://users.iems.northwestern.edu/ nocedal/lbfgs.html
[43] Nvidia, Whitepaper. NVIDIAs Next Generation CUDA Computer Architecture: Kepler GK110, Nvidia, 2012.; Nvidia, Whitepaper. NVIDIAs Next Generation CUDA Computer Architecture: Kepler GK110, Nvidia, 2012.
[44] Pardalos, P. M.; Romeijn, H. E., Handbook of Global Optimization (2002), Springer · Zbl 0991.00017
[45] Phelps, R., Minimization of functions having lipschitz continuous first partial derivatives, Pacific J. Math., 16, 1-3 (1966) · Zbl 0202.46105
[46] Phelps, R., The gradient projection method using currys steplength, SIAM J. Control Optim., 24, 692-699 (1986) · Zbl 0603.90117
[47] Polak, E.; Ribière, G., Note sur la convergence de méthodes des directions conjuguées, Rev. Fr. Inform. Rech. Oper., 3e Année 16, 35-43 (1969) · Zbl 0174.48001
[48] Powell, M., An efficient method for finding the minimum of a function of several variables without calculating derivatives, Comput. J., 7, 155-162 (1964) · Zbl 0132.11702
[49] Pulkkinen, S., A Review of Methods for Unconstrained Optimization: Theory, Implementation and Testing (2008), University of Helsinki
[50] Salomon, R., Reevaluating genetic algorithms performance under coordinate rotation of benchmark functions, Biosystems, 39, 263-278 (1995)
[51] Shanno, D. F., Conditioning of quasi-newton methods for function minimization, Math. Comp., 24, 647-650 (1970) · Zbl 0225.65073
[52] Spendley, W.; Hext, G. R.; Himsworth, F. R., Sequential application of simplex designs in optimization and evolutionary operation, Technometrics, 4, 441-461 (1962) · Zbl 0121.35603
[53] Vaz, A. I.F.; Vicente, L. N., A particle swarm pattern search method for bound constrained global optimization, Int. J. Comput. Math., 39, 2, 197-219 (2007) · Zbl 1180.90252
[54] Vaz, A. I.F.; Vicente, L. N., Pswarm: a hybrid solver for linearly constrained global derivative-free optimization, Optim. Methods Softw., 24, 4-5, 669-685 (2009) · Zbl 1177.90327
[55] Wales, D. J.; Doye, J. P.K., Global optimization by Basin-Hopping and the lowest energy structures of Lennard-Jones clusters containing up to 110 atoms, J. Phys. Chem. A, 101, 5111-5116 (1997)
[56] Wales, D. J.; Doye, J. P.K., Energy Landscapes (2003), Cambridge University Press
[57] D.J. Wales, J.P.K. Doye, A. Dullweber, M.P. Hodges, F.Y.N.F. Calvo, J. Hernández-Rojas, T.F. Middleton, The Cambridge Cluster Database. URL http://www-wales.ch.cam.ac.uk/CCD.html; D.J. Wales, J.P.K. Doye, A. Dullweber, M.P. Hodges, F.Y.N.F. Calvo, J. Hernández-Rojas, T.F. Middleton, The Cambridge Cluster Database. URL http://www-wales.ch.cam.ac.uk/CCD.html
[58] Wales, D. J.; Scheraga, H. A., Global optimization of clusters, crystals, and biomolecules, Science, 285, 1368-1372 (1999)
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. In some cases that data have been complemented/enhanced by data from zbMATH Open. This attempts to reflect the references listed in the original paper as accurately as possible without claiming completeness or a perfect matching.