×

A new taxonomy of global optimization algorithms. (English) Zbl 1530.90116

Summary: Surrogate-based optimization, nature-inspired metaheuristics, and hybrid combinations have become state of the art in algorithm design for solving real-world optimization problems. Still, it is difficult for practitioners to get an overview that explains their advantages in comparison to a large number of available methods in the scope of optimization. Available taxonomies lack the embedding of current approaches in the larger context of this broad field. This article presents a taxonomy of the field, which explores and matches algorithm strategies by extracting similarities and differences in their search strategies. A particular focus lies on algorithms using surrogates, nature-inspired designs, and those created by automatic algorithm generation. The extracted features of algorithms, their main concepts, and search operators, allow us to create a set of classification indicators to distinguish between a small number of classes. The features allow a deeper understanding of components of the search strategies and further indicate the close connections between the different algorithm designs. We present intuitive analogies to explain the basic principles of the search algorithms, particularly useful for novices in this research field. Furthermore, this taxonomy allows recommendations for the applicability of the corresponding algorithms.

MSC:

90C59 Approximation methods and heuristics in mathematical programming
68W50 Evolutionary algorithms, genetic algorithms (computational aspects)
90C26 Nonconvex programming, global optimization

References:

[1] Angeline PJ (1995) Adaptive and self-adaptive evolutionary computations. In: Computational intelligence: a dynamic systems perspective. Citeseer
[2] Archetti, F.; Schoen, F., A survey on the global optimization problem: general theory and computational approaches, Ann Oper Res, 1, 2, 87-110 (1984) · Zbl 0671.90057 · doi:10.1007/BF01876141
[3] Arnold, DV; Beyer, HG, A comparison of evolution strategies with other direct search methods in the presence of noise, Comput Optim Appl, 24, 1, 135-159 (2003) · Zbl 1035.90110 · doi:10.1023/A:1021810301763
[4] Arnold DV, Hansen N (2012) A (1 + 1)-CMA-ES for constrained optimisation. In: Proceedings of the 14th annual conference on genetic and evolutionary computation. ACM, pp 297-304
[5] Arora, J.; Elwakeil, O.; Chahande, A.; Hsieh, C., Global optimization methods for engineering applications: a review, Struct Optim, 9, 3-4, 137-159 (1995) · doi:10.1007/BF01743964
[6] Audet C (2014) A survey on direct search methods for blackbox optimization and their applications. In: Mathematics without boundaries. Springer, pp 31-56 · Zbl 1321.90125
[7] Augusto DA, Barbosa HJ (2000) Symbolic regression via genetic programming. In: Sixth Brazilian symposium on neural networks, 2000. Proceedings. IEEE, pp 173-178
[8] Back, T., Evolutionary algorithms in theory and practice: evolution strategies, evolutionary programming, genetic algorithms (1996), Oxford: Oxford University Press, Oxford · Zbl 0877.68060 · doi:10.1093/oso/9780195099713.001.0001
[9] Back, T.; Fogel, DB; Michalewicz, Z., Handbook of evolutionary computation (1997), Bristol: IOP Publishing Ltd, Bristol · Zbl 0883.68001 · doi:10.1887/0750308958
[10] Baluja S (1994) Population-based incremental learning: a method for integrating genetic search based function optimization and competitive learning
[11] Bartz-Beielstein T (2010) SPOT: an R package for automatic and interactive tuning of optimization algorithms by sequential parameter optimization. arXiv preprint arXiv:10064645
[12] Bartz-Beielstein, T.; Zaefferer, M., Model-based methods for continuous and discrete global optimization, Appl Soft Comput, 55, 154-167 (2017) · doi:10.1016/j.asoc.2017.01.039
[13] Bartz-Beielstein T, Lasarczyk CW, Preuß M (2005) Sequential parameter optimization. In: The 2005 IEEE Congress on evolutionary computation, 2005, vol 1. IEEE, pp 773-780 · Zbl 1208.90166
[14] Beume, N.; Naujoks, B.; Emmerich, M., SMS-EMOA: multiobjective selection based on dominated hypervolume, Eur J Oper Res, 181, 3, 1653-1669 (2007) · Zbl 1123.90064 · doi:10.1016/j.ejor.2006.08.008
[15] Beyer, HG, The theory of evolution strategies (2013), Berlin: Springer, Berlin
[16] Beyer, HG; Schwefel, HP, Evolution strategies: a comprehensive introduction, Nat Comput, 1, 1, 3-52 (2002) · Zbl 1014.68134 · doi:10.1023/A:1015059928466
[17] Bezerra LC, López-Ibánez M, Stützle T (2014) Automatic design of evolutionary algorithms for multi-objective combinatorial optimization. In: International conference on parallel problem solving from nature. Springer, pp 508-517
[18] Blum, C.; Puchinger, J.; Raidl, GR; Roli, A., Hybrid metaheuristics in combinatorial optimization: a survey, Appl Soft Comput, 11, 6, 4135-4151 (2011) · Zbl 1205.90298 · doi:10.1016/j.asoc.2011.02.032
[19] Booker, AJ; Dennis, J. Jr; Frank, PD; Serafini, DB; Torczon, V.; Trosset, MW, A rigorous framework for optimization of expensive functions by surrogates, Struct Optim, 17, 1, 1-13 (1999) · doi:10.1007/BF01197708
[20] Bosman PAN, Thierens D (2000) Continuous iterated density estimation evolutionary algorithms within the idea framework
[21] Bossek J, Doerr C, Kerschke P (2020) Initial design strategies and their effects on sequential model-based optimization: an exploratory case study based on BBOB. In: Proceedings of the 2020 genetic and evolutionary computation conference, GECCO ’20. Association for Computing Machinery, New York, pp 778-786. doi:10.1145/3377930.3390155
[22] Bouhlel, MA; Hwang, JT; Bartoli, N.; Lafage, R.; Morlier, J.; Martins, JR, A python surrogate modeling framework with derivatives, Adv Eng Softw, 135, 102662 (2019) · doi:10.1016/j.advengsoft.2019.03.005
[23] Boussaïd, I.; Lepagnot, J.; Siarry, P., A survey on optimization metaheuristics, Inf Sci, 237, 82-117 (2013) · Zbl 1321.90156 · doi:10.1016/j.ins.2013.02.041
[24] Breiman, L., Random forests, Mach Learn, 45, 1, 5-32 (2001) · Zbl 1007.68152 · doi:10.1023/A:1010933404324
[25] Breiman, L.; Friedman, J.; Stone, CJ; Olshen, RA, Classification and regression trees (1984), Boca Raton: CRC Press, Boca Raton · Zbl 0541.62042
[26] Burke E, Kendall G, Newall J, Hart E, Ross P, Schulenburg S (2003) Hyper-heuristics: an emerging direction in modern search technology. In: Handbook of metaheuristics. Springer, pp 457-474 · Zbl 1102.90377
[27] Burke EK, Hyde M, Kendall G, Ochoa G, Özcan E, Woodward JR (2010) A classification of hyper-heuristic approaches. In: Handbook of metaheuristics. Springer, pp 449-468
[28] Burke, EK; Gendreau, M.; Hyde, M.; Kendall, G.; Ochoa, G.; Özcan, E.; Qu, R., Hyper-heuristics: a survey of the state of the art, J Oper Res Soc, 64, 12, 1695-1724 (2013) · doi:10.1057/jors.2013.71
[29] Carson Y, Maria A (1997) Simulation optimization: methods and applications. In: Proceedings of the 29th conference on Winter simulation. IEEE Computer Society, pp 118-126
[30] Chelouah, R.; Siarry, P., Tabu search applied to global optimization, Eur J Oper Res, 123, 2, 256-270 (2000) · Zbl 0961.90037 · doi:10.1016/S0377-2217(99)00255-6
[31] Coello, CAC, Theoretical and numerical constraint-handling techniques used with evolutionary algorithms: a survey of the state of the art, Comput Methods Appl Mech Eng, 191, 11, 1245-1287 (2002) · Zbl 1026.74056 · doi:10.1016/S0045-7825(01)00323-1
[32] Collobert R, Weston J (2008) A unified architecture for natural language processing: deep neural networks with multitask learning. In: Proceedings of the 25th international conference on Machine learning. ACM, pp 160-167
[33] Corana, A.; Marchesi, M.; Martini, C.; Ridella, S., Minimizing multimodal functions of continuous variables with the “simulated annealing” algorithm, ACM Trans Math Softw (TOMS), 13, 3, 262-280 (1987) · Zbl 0632.65075 · doi:10.1145/29380.29864
[34] Cowling P, Kendall G, Soubeiga E (2000) A hyperheuristic approach to scheduling a sales summit. In: International conference on the practice and theory of automated timetabling. Springer, pp 176-190 · Zbl 0982.68516
[35] Cowling P, Kendall G, Soubeiga E (2002) Hyperheuristics: a tool for rapid prototyping in scheduling and optimisation. In: Workshops on applications of evolutionary computation. Springer, pp 1-10 · Zbl 1044.68749
[36] Cox DD, John S (1997) SDO: a statistical method for global optimization. In: Multidisciplinary design optimization: state of the art, pp 315-329
[37] Črepinšek, M.; Liu, SH; Mernik, M., Exploration and exploitation in evolutionary algorithms: a survey, ACM Comput Surv (CSUR), 45, 3, 35 (2013) · Zbl 1293.68251 · doi:10.1145/2480741.2480752
[38] Crombecq, K.; Gorissen, D.; Deschrijver, D.; Dhaene, T., A novel hybrid sequential design strategy for global surrogate modeling of computer experiments, SIAM J Sci Comput, 33, 4, 1948-1974 (2011) · Zbl 1227.62059 · doi:10.1137/090761811
[39] Deb, K.; Pratap, A.; Agarwal, S.; Meyarivan, T., A fast and elitist multiobjective genetic algorithm: NSGA-II, IEEE Trans Evol Comput, 6, 2, 182-197 (2002) · doi:10.1109/4235.996017
[40] Doerr, B.; Doerr, C.; Yang, J., Optimal parameter choices via precise black-box analysis, Theor Comput Sci, 801, 1-34 (2020) · Zbl 1436.68408 · doi:10.1016/j.tcs.2019.06.014
[41] Dorigo, M.; Birattari, M.; Stutzle, T., Ant colony optimization, IEEE Comput Intell Mag, 1, 4, 28-39 (2006) · doi:10.1109/MCI.2006.329691
[42] Duchi, J.; Hazan, E.; Singer, Y., Adaptive subgradient methods for online learning and stochastic optimization, J Mach Learn Res, 12, 7, 2121-2159 (2011) · Zbl 1280.68164
[43] Eiben, AE; Smith, JE, Introduction to evolutionary computing (2003), Berlin: Springer, Berlin · Zbl 1028.68022 · doi:10.1007/978-3-662-05094-1
[44] Eiben, AE; Smith, JE, Introduction to evolutionary computing (2015), Berlin: Springer, Berlin · Zbl 1327.68003 · doi:10.1007/978-3-662-44874-8
[45] Eiben, AE; Hinterding, R.; Michalewicz, Z., Parameter control in evolutionary algorithms, IEEE Trans Evolut Comput, 3, 2, 124-141 (1999) · doi:10.1109/4235.771166
[46] Emmerich, MT; Giannakoglou, KC; Naujoks, B., Single-and multiobjective evolutionary optimization assisted by Gaussian random field metamodels, IEEE Trans Evolut Comput, 10, 4, 421-439 (2006) · doi:10.1109/TEVC.2005.859463
[47] Faber, R.; Jockenhövel, T.; Tsatsaronis, G., Dynamic optimization with simulated annealing, Comput Chem Eng, 29, 2, 273-290 (2005) · doi:10.1016/j.compchemeng.2004.08.020
[48] Filipiak P, Lipinski P (2014) Infeasibility driven evolutionary algorithm with feed-forward prediction strategy for dynamic constrained optimization problems. In: Applications of evolutionary computation. Springer, pp 817-828
[49] Flasch O, Mersmann O, Bartz-Beielstein T (2010) RGP: an open source genetic programming system for the R environment. In: Proceedings of the 12th annual conference companion on genetic and evolutionary computation, GECCO ’10. ACM, New York, pp 2071-2072. doi:10.1145/1830761.1830867
[50] Fleming, PJ; Purshouse, RC, Evolutionary algorithms in control systems engineering: a survey, Control Eng Pract, 10, 11, 1223-1241 (2002) · doi:10.1016/S0967-0661(02)00081-3
[51] Fletcher R (1976) Conjugate gradient methods for indefinite systems. In: Numerical analysis. Springer, pp 73-89 · Zbl 0326.65033
[52] Fogel, LJ; Owens, AJ; Walsh, MJ, Artificial intelligence through simulated evolution (1966), Hoboken: Wiley, Hoboken · Zbl 0148.40701
[53] Fomin, FV; Kaski, P., Exact exponential algorithms, Commun ACM, 56, 3, 80-88 (2013) · doi:10.1145/2428556.2428575
[54] Fonseca, CM; Fleming, PJ, Genetic algorithms for multiobjective optimization: formulation, discussion and generalization, ICGA, 93, 416-423 (1993)
[55] Forrester, AI; Keane, AJ, Recent advances in surrogate-based optimization, Prog Aerosp Sci, 45, 1, 50-79 (2009) · doi:10.1016/j.paerosci.2008.11.001
[56] Forrester, A.; Sobester, A.; Keane, A., Engineering design via surrogate modelling: a practical guide (2008), Hoboken: Wiley, Hoboken · doi:10.1002/9780470770801
[57] Freund, Y.; Schapire, RE, A decision-theoretic generalization of on-line learning and an application to boosting, J Comput Syst Sci, 55, 1, 119-139 (1997) · Zbl 0880.68103 · doi:10.1006/jcss.1997.1504
[58] Friese M, Bartz-Beielstein T, Emmerich MTM (2016) Building ensembles of surrogates by optimal convex combination. In: Mernik M, Papa G (eds) Bioinspired optimization methods and their applications, pp 131-144
[59] Gaier A, Asteroth A, Mouret JB (2018) Data-efficient neuroevolution with kernel-based surrogate models. In: Proceedings of the genetic and evolutionary computation conference, pp 85-92
[60] Gallagher, M.; Frean, MR; Downs, T., Real-valued evolutionary optimization using a flexible probability density estimator, GECCO, 99, 840-846 (1999)
[61] Gämperle, R.; Müller, SD; Koumoutsakos, P., A parameter study for differential evolution, Adv Intell Syst Fuzzy Syst Evolut Comput, 10, 293-298 (2002)
[62] Glover, F., Tabu search—part I, ORSA J Comput, 1, 3, 190-206 (1989) · Zbl 0753.90054 · doi:10.1287/ijoc.1.3.190
[63] Goel, T.; Haftka, RT; Shyy, W.; Queipo, NV, Ensemble of surrogates, Struct Multidiscip Optim, 33, 3, 199-216 (2007) · doi:10.1007/s00158-006-0051-9
[64] Goffe, WL; Ferrier, GD; Rogers, J., Global optimization of statistical functions with simulated annealing, J Econom, 60, 1, 65-99 (1994) · Zbl 0789.62095 · doi:10.1016/0304-4076(94)90038-8
[65] Guo H (2003) A Bayesian approach for automatic algorithm selection. In: Proceedings of the International Joint Conference on Artificial Intelligence (IJCAI03), workshop on AI and autonomic computing, Acapulco, Mexico, pp 1-5
[66] Haftka, RT; Villanueva, D.; Chaudhuri, A., Parallel surrogate-assisted global optimization with expensive functions—a survey, Struct Multidiscip Optim, 54, 1, 3-13 (2016) · doi:10.1007/s00158-016-1432-3
[67] Hansen N, Auger A, Finck S, Ros R (2010) Real-parameter black-box optimization benchmarking 2010: experimental setup. Research report no: RR-7215
[68] Hansen, P.; Mladenovic, N., Variable neighbourhood search. Handbook of metaheuristics (2003), Dordrecht: Kluwer Academic Publishers, Dordrecht · Zbl 1029.90062
[69] Hansen, N.; Müller, SD; Koumoutsakos, P., Reducing the time complexity of the derandomized evolution strategy with covariance matrix adaptation (CMA-ES), Evolut Comput, 11, 1, 1-18 (2003) · doi:10.1162/106365603321828970
[70] Hansen N, Auger A, Finck S, Ros R (2010a) Real-parameter black-box optimization benchmarking 2010: experimental setup
[71] Hansen N, Auger A, Ros R, Finck S, Pošík P (2010b) Comparing results of 31 algorithms from the black-box optimization benchmarking bbob-2009. In: Proceedings of the 12th annual conference companion on Genetic and evolutionary computation. ACM, pp 1689-1696
[72] Hansen, P.; Mladenović, N.; Pérez, JAM, Variable neighbourhood search: methods and applications, Ann Oper Res, 175, 1, 367-407 (2010) · Zbl 1185.90211 · doi:10.1007/s10479-009-0657-6
[73] Harik G et al (1999) Linkage learning via probabilistic modeling in the ECGA. IlliGAL report 99010
[74] Hauschild, M.; Pelikan, M., An introduction and survey of estimation of distribution algorithms, Swarm Evolut Comput, 1, 3, 111-128 (2011) · doi:10.1016/j.swevo.2011.08.003
[75] Haykin, S., A comprehensive foundation, Neural Netw, 2, 41 (2004)
[76] Henderson D, Jacobson SH, Johnson AW (2003) The theory and practice of simulated annealing. In: Handbook of metaheuristics. Springer, pp 287-319 · Zbl 1102.90381
[77] Hinton, GE; Osindero, S.; Teh, YW, A fast learning algorithm for deep belief nets, Neural Comput, 18, 7, 1527-1554 (2006) · Zbl 1106.68094 · doi:10.1162/neco.2006.18.7.1527
[78] Hinton, G.; Deng, L.; Yu, D.; Dahl, GE; Ar, M.; Jaitly, N.; Senior, A.; Vanhoucke, V.; Nguyen, P.; Sainath, TN, Deep neural networks for acoustic modeling in speech recognition: the shared views of four research groups, IEEE Signal Process Mag, 29, 6, 82-97 (2012) · doi:10.1109/MSP.2012.2205597
[79] Holland JH (1992) Adaptation in natural and artificial systems: an introductory analysis with applications to biology, control, and artificial intelligence. MIT Press
[80] Hornik, K.; Stinchcombe, M.; White, H., Multilayer feedforward networks are universal approximators, Neural Netw, 2, 5, 359-366 (1989) · Zbl 1383.92015 · doi:10.1016/0893-6080(89)90020-8
[81] Hu, N., Tabu search method with random moves for globally optimal design, Int J Numer Methods Eng, 35, 5, 1055-1070 (1992) · doi:10.1002/nme.1620350508
[82] Hutter, F.; Hoos, HH; Leyton-Brown, K., Sequential model-based optimization for general algorithm configuration, LION, 5, 507-523 (2011)
[83] Jaderberg M, Dalibard V, Osindero S, Czarnecki WM, Donahue J, Razavi A, Vinyals O, Green T, Dunning I, Simonyan K et al (2017) Population based training of neural networks. arXiv preprint arXiv:171109846
[84] Jin, Y., A comprehensive survey of fitness approximation in evolutionary computation, Soft Comput Fusion Found Methodol Appl, 9, 1, 3-12 (2005)
[85] Jin, Y., Surrogate-assisted evolutionary computation: recent advances and future challenges, Swarm Evolut Comput, 1, 2, 61-70 (2011) · doi:10.1016/j.swevo.2011.05.001
[86] Jin, Y.; Branke, J., Evolutionary optimization in uncertain environments—a survey, IEEE Trans Evolut Comput, 9, 3, 303-317 (2005) · doi:10.1109/TEVC.2005.846356
[87] Jones, DR, A taxonomy of global optimization methods based on response surfaces, J Glob Optim, 21, 4, 345-383 (2001) · Zbl 1172.90492 · doi:10.1023/A:1012771025575
[88] Jones, DR; Schonlau, M.; Welch, WJ, Efficient global optimization of expensive black-box functions, J Glob Optim, 13, 4, 455-492 (1998) · Zbl 0917.90270 · doi:10.1023/A:1008306431147
[89] Kennedy J, Eberhart R (1995) Particle swarm optimization. In: IEEE international conference on neural networks, 1995. Proceedings, vol 4. IEEE, pp 1942-1948
[90] Kerschke, P.; Trautmann, H., Automated algorithm selection on continuous black-box problems by combining exploratory landscape analysis and machine learning, Evolut Comput, 27, 1, 99-127 (2019) · doi:10.1162/evco_a_00236
[91] Khan N, Goldberg DE, Pelikan M (2002) Multi-objective Bayesian optimization algorithm. In: Proceedings of the 4th annual conference on genetic and evolutionary computation, GECCO’02. Morgan Kaufmann Publishers Inc., San Francisco, p 684. http://dl.acm.org/citation.cfm?id=2955491.2955599
[92] Kingma DP, Ba J (2014) Adam: a method for stochastic optimization. arXiv preprint arXiv:14126980
[93] Kirkpatrick, S.; Gelatt, CD; Vecchi, MP, Optimization by simulated annealing, Science, 220, 4598, 671-680 (1983) · Zbl 1225.90162 · doi:10.1126/science.220.4598.671
[94] Kolda, TG; Lewis, RM; Torczon, V., Optimization by direct search: new perspectives on some classical and modern methods, SIAM Rev, 45, 3, 385-482 (2003) · Zbl 1059.90146 · doi:10.1137/S003614450242889
[95] Koza, JR, Genetic programming: on the programming of computers by means of natural selection (1992), Cambridge: MIT Press, Cambridge · Zbl 0850.68161
[96] Kushner, HJ, A new method of locating the maximum point of an arbitrary multipeak curve in the presence of noise, J Basic Eng, 86, 1, 97-106 (1964) · doi:10.1115/1.3653121
[97] Larrañaga, P.; Lozano, JA, Estimation of distribution algorithms: a new tool for evolutionary computation (2001), Berlin: Springer, Berlin · Zbl 0979.00024
[98] Larrañaga P, Etxeberria R, Lozano JA, Peña JM (1999) Optimization by learning and simulation of Bayesian and Gaussian networks
[99] Lawler, EL; Wood, DE, Branch-and-bound methods: a survey, Oper Res, 14, 4, 699-719 (1966) · Zbl 0143.42501 · doi:10.1287/opre.14.4.699
[100] Leon, A., A classified bibliography on optimization. Recent advances in optimization techniques, 599-649 (1966), New York: Wiley, New York
[101] Lewis, RM; Torczon, V.; Trosset, MW, Direct search methods: then and now, J Comput Appl Math, 124, 1, 191-207 (2000) · Zbl 0969.65055 · doi:10.1016/S0377-0427(00)00423-4
[102] Lim, D.; Jin, Y.; Ong, YS; Sendhoff, B., Generalizing surrogate-assisted evolutionary computation, IEEE Trans Evolut Comput, 14, 3, 329 (2010) · doi:10.1109/TEVC.2009.2027359
[103] Lindauer, M.; Hoos, HH; Hutter, F.; Schaub, T., Autofolio: an automatically configured algorithm selector, J Artif Intell Res, 53, 745-778 (2015) · doi:10.1613/jair.4726
[104] Liu, DC; Nocedal, J., On the limited memory BFGS method for large scale optimization, Math Program, 45, 1-3, 503-528 (1989) · Zbl 0696.90048 · doi:10.1007/BF01589116
[105] Lizotte DJ (2008) Practical Bayesian optimization. University of Alberta
[106] Lobo, F.; Lima, CF; Michalewicz, Z., Parameter setting in evolutionary algorithms (2007), Berlin: Springer, Berlin · Zbl 1139.68047 · doi:10.1007/978-3-540-69432-8
[107] Locatelli M (2002) Simulated annealing algorithms for continuous global optimization. In: Handbook of global optimization. Springer, pp 179-229 · Zbl 1111.90361
[108] López-Ibáñez, M.; Dubois-Lacoste, J.; Cáceres, LP; Birattari, M.; Stützle, T., The irace package: iterated racing for automatic algorithm configuration, Oper Res Perspect, 3, 43-58 (2016)
[109] Loshchilov I, Schoenauer M, Sebag M (2012) Self-adaptive surrogate-assisted covariance matrix adaptation evolution strategy. In: Proceedings of the 14th annual conference on genetic and evolutionary computation. ACM, pp 321-328
[110] Marler, RT; Arora, JS, Survey of multi-objective optimization methods for engineering, Struct Multidiscip Optim, 26, 6, 369-395 (2004) · Zbl 1243.90199 · doi:10.1007/s00158-003-0368-6
[111] Marsden, AL; Wang, M.; Dennis, JE Jr; Moin, P., Optimal aeroacoustic shape design using the surrogate management framework, Optim Eng, 5, 2, 235-262 (2004) · Zbl 1116.90417 · doi:10.1023/B:OPTE.0000033376.89159.65
[112] Martin MA, Tauritz DR (2013) Evolving black-box search algorithms employing genetic programming. In: Proceedings of the 15th annual conference companion on genetic and evolutionary computation, pp 1497-1504
[113] McKay B, Willis MJ, Barton GW (1995) Using a tree structured genetic algorithm to perform symbolic regression. In: First international conference on genetic algorithms in engineering systems: innovations and applications, 1995. GALESIA. (Conf. Publ. No. 414). IET, pp 487-492
[114] Mercer, RE; Sampson, J., Adaptive search using a reproductive meta-plan, Kybernetes, 7, 3, 215-228 (1978) · doi:10.1108/eb005486
[115] Mersmann O, Bischl B, Trautmann H, Preuss M, Weihs C, Rudolph G (2011) Exploratory landscape analysis. In: Proceedings of the 13th annual conference on genetic and evolutionary computation, GECCO ’11. Association for Computing Machinery, New York, pp 829-836. doi:10.1145/2001576.2001690
[116] Mladenović, N.; Dražić, M.; Kovačevic-Vujčić, V.; Čangalović, M., General variable neighborhood search for the continuous optimization, Eur J Oper Res, 191, 3, 753-770 (2008) · Zbl 1156.90005 · doi:10.1016/j.ejor.2006.12.064
[117] Mockus J (1974) On Bayesian methods for seeking the extremum. In: Proceedings of the IFIP Technical Conference. Springer, pp 400-404 · Zbl 0311.90042
[118] Mockus, J., Application of Bayesian approach to numerical methods of global and stochastic optimization, J Glob Optim, 4, 4, 347-365 (1994) · Zbl 0801.90099 · doi:10.1007/BF01099263
[119] Mockus, J., Bayesian approach to global optimization: theory and applications (2012), Berlin: Springer, Berlin · Zbl 0693.49001
[120] Molina, D.; Lozano, M.; García-Martínez, C.; Herrera, F., Memetic algorithms for continuous optimisation based on local search chains, Evolut Comput, 18, 1, 27-63 (2010) · doi:10.1162/evco.2010.18.1.18102
[121] Montgomery, DC; Montgomery, DC; Montgomery, DC, Design and analysis of experiments (1984), New York: Wiley, New York · Zbl 0547.76118
[122] Moscato P et al (1989) On evolution, search, optimization, genetic algorithms and martial arts: Towards memetic algorithms. Caltech concurrent computation program, C3P Report 826:1989
[123] Müller, J., MISO: mixed-integer surrogate optimization framework, Optim Eng, 17, 1, 177-203 (2016) · Zbl 1364.90230 · doi:10.1007/s11081-015-9281-2
[124] Müller, J.; Shoemaker, CA, Influence of ensemble surrogate models and sampling strategy on the solution quality of algorithms for computationally expensive black-box global optimization problems, J Glob Optim, 60, 2, 123-144 (2014) · Zbl 1312.90064 · doi:10.1007/s10898-014-0184-0
[125] Naujoks B, Beume N, Emmerich M (2005) Multi-objective optimisation using s-metric selection: application to three-dimensional solution spaces. In: The 2005 IEEE Congress on Evolutionary Computation, vol 2. IEEE, pp 1282-1289 · Zbl 1109.68595
[126] Nelder, JA; Mead, R., A simplex method for function minimization, Comput J, 7, 4, 308-313 (1965) · Zbl 0229.65053 · doi:10.1093/comjnl/7.4.308
[127] Neumaier, A., Complete search in continuous global optimization and constraint satisfaction, Acta Numer, 13, 271-369 (2004) · Zbl 1113.90124 · doi:10.1017/S0962492904000194
[128] Ong YS, Nair P, Keane A, Wong K (2005) Surrogate-assisted evolutionary optimization frameworks for high-fidelity engineering design problems. In: Knowledge Incorporation in Evolutionary Computation. Springer, pp 307-331
[129] Papa G, Doerr C (2020) Dynamic control parameter choices in evolutionary computation: Gecco 2020 tutorial. In: Proceedings of the 2020 genetic and evolutionary computation conference companion, pp 927-956
[130] Pearl J (1985) Heuristics. intelligent search strategies for computer problem solving. The Addison-Wesley series in artificial intelligence. Addison-Wesley, Reading, Reprinted version
[131] Pittenger, AO, An introduction to quantum computing algorithms (2012), Berlin: Springer, Berlin · Zbl 0940.81004
[132] Queipo, NV; Haftka, RT; Shyy, W.; Goel, T.; Vaidyanathan, R.; Tucker, PK, Surrogate-based analysis and optimization, Prog Aerosp Sci, 41, 1, 1-28 (2005) · doi:10.1016/j.paerosci.2005.02.001
[133] Rechenberg I (1973) Evolutionsstrategie—Optimierung technischer Systeme nach Prinzipien der biologischen Evolution. Frommann-Holzboog
[134] Rechenberg I (1994) Evolutionsstrategie ’94. Frommann-Holzboog
[135] Regis, RG; Shoemaker, CA, Combining radial basis function surrogates and dynamic coordinate search in high-dimensional expensive black-box optimization, Eng Optim, 45, 5, 529-555 (2013) · doi:10.1080/0305215X.2012.687731
[136] Rehbach F, Zaefferer M, Stork J, Bartz-Beielstein T (2018) Comparison of parallel surrogate-assisted optimization approaches. In: Proceedings of the genetic and evolutionary computation conference. ACM, pp 1348-1355
[137] Richter SN, Tauritz DR (2018) The automated design of probabilistic selection methods for evolutionary algorithms. In: Proceedings of the genetic and evolutionary computation conference companion, pp 1545-1552
[138] Rozenberg, G.; Bck, T.; Kok, JN, Handbook of natural computing (2011), Berlin: Springer, Berlin
[139] Ruder S (2016) An overview of gradient descent optimization algorithms. arXiv preprint arXiv:160904747
[140] Sacks, J.; Welch, WJ; Mitchell, TJ; Wynn, HP, Design and analysis of computer experiments, Stat Sci, 4, 409-423 (1989) · Zbl 0955.62619
[141] Santana R (2017) Gray-box optimization and factorized distribution algorithms: where two worlds collide. arXiv preprint arXiv:170703093
[142] Schuman CD, Potok TE, Patton RM, Birdwell JD, Dean ME, Rose GS, Plank JS (2017) A survey of neuromorphic computing and neural networks in hardware. arXiv preprint arXiv:170506963
[143] Schwefel, HP, Numerische Optimierung von Computer-Modellen mittels der Evolutionsstrategie: mit einer vergleichenden Einführung in die Hill-Climbing-und Zufallsstrategie (1977), Basel: Birkhäuser, Basel · Zbl 0347.65035 · doi:10.1007/978-3-0348-5927-1
[144] Schwefel, HPP, Evolution and optimum seeking: the sixth generation (1993), Hoboken: Wiley, Hoboken
[145] Shanno, DF, Conditioning of quasi-Newton methods for function minimization, Math Comput, 24, 111, 647-656 (1970) · Zbl 0225.65073 · doi:10.1090/S0025-5718-1970-0274029-X
[146] Shi Y, Eberhart R (1998) A modified particle swarm optimizer. In: The 1998 IEEE international conference on evolutionary computation proceedings, 1998. IEEE World Congress on Computational Intelligence. IEEE, pp 69-73
[147] Shir OM, Bäck T (2005) Niching in evolution strategies. In: Proceedings of the 7th annual conference on genetic and evolutionary computation. ACM, pp 915-916
[148] Siarry, P.; Berthiau, G., Fitting of tabu search to optimize functions of continuous variables, Int J Numer Methods Eng, 40, 13, 2449-2457 (1997) · Zbl 0882.65049 · doi:10.1002/(SICI)1097-0207(19970715)40:13<2449::AID-NME172>3.0.CO;2-O
[149] Siarry, P.; Berthiau, G.; Durdin, F.; Haussy, J., Enhanced simulated annealing for globally minimizing functions of many-continuous variables, ACM Trans Math Softw (TOMS), 23, 2, 209-228 (1997) · Zbl 0887.65067 · doi:10.1145/264029.264043
[150] Smit S, Eiben A (2011) Multi-problem parameter tuning using bonesa. In: Artificial evolution, pp 222-233
[151] Snoek J, Larochelle H, Adams RP (2012) Practical Bayesian optimization of machine learning algorithms. In: Advances in neural information processing systems, pp 2951-2959
[152] Søndergaard J, Madsen K, Nielsen HB (2003) Optimization using surrogate models-by the space mapping technique. Ph.D. thesis, Technical University of Denmark
[153] Stanley, KO; Clune, J.; Lehman, J.; Miikkulainen, R., Designing neural networks through neuroevolution, Nat Mach Intell, 1, 1, 24-35 (2019) · doi:10.1038/s42256-018-0006-z
[154] Stork J, Zaefferer M, Bartz-Beielstein T, Eiben A (2019) Surrogate models for enhancing the efficiency of neuroevolution in reinforcement learning. In: Proceedings of the genetic and evolutionary computation conference, pp 934-942
[155] Swersky K, Snoek J, Adams RP (2013) Multi-task Bayesian optimization. In: Advances in neural information processing systems, pp 2004-2012
[156] Talbi, EG, A taxonomy of hybrid metaheuristics, J Heuristics, 8, 5, 541-564 (2002) · doi:10.1023/A:1016540724870
[157] Talbi, EG, Metaheuristics: from design to implementation (2009), Hoboken: Wiley, Hoboken · Zbl 1190.90293 · doi:10.1002/9780470496916
[158] Törn, A.; Zilinskas, A., Global optimization (1989), Berlin: Springer, Berlin · Zbl 0752.90075 · doi:10.1007/3-540-50871-6
[159] Van Groenigen, J.; Stein, A., Constrained optimization of spatial sampling using continuous simulated annealing, J Environ Qual, 27, 5, 1078-1086 (1998) · doi:10.2134/jeq1998.00472425002700050013x
[160] van Rijn S, Wang H, van Stein B, Bäck T (2017) Algorithm configuration data mining for CMA evolution strategies. In: Proceedings of the genetic and evolutionary computation conference, GECCO ’17. ACM, New York, pp 737-744. doi:10.1145/3071178.3071205
[161] Vermetten D, van Rijn S, Bäck T, Doerr C (2019) Online selection of CMA-ES variants. In: Proceedings of the genetic and evolutionary computation conference, GECCO ’19. ACM, New York, pp 951-959. doi:10.1145/3321707.3321803
[162] Whitley, LD; Chicano, F.; Goldman, BW, Gray box optimization for Mk landscapes (Nk landscapes and MAX-KSAT), Evolut Comput, 24, 3, 491-519 (2016) · doi:10.1162/EVCO_a_00184
[163] Woeginger GJ (2003) Exact algorithms for NP-hard problems: a survey. In: Combinatorial optimization—eureka, you shrink!. Springer, pp 185-207 · Zbl 1024.68529
[164] Won KS, Ray T (2004) Performance of kriging and cokriging based surrogate models within the unified framework for surrogate assisted optimization. In: Congress on evolutionary computation, 2004. CEC2004, vol 2. IEEE, pp 1577-1585
[165] Zaefferer M, Stork J, Friese M, Fischbach A, Naujoks B, Bartz-Beielstein T (2014) Efficient global optimization for combinatorial problems. In: Proceedings of the 2014 annual conference on genetic and evolutionary computation, pp 871-878
[166] Zlochin, M.; Birattari, M.; Meuleau, N.; Dorigo, M., Model-based search for combinatorial optimization: a critical survey, Ann Oper Res, 131, 1-4, 373-395 (2004) · Zbl 1067.90162 · doi:10.1023/B:ANOR.0000039526.52305.af
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. In some cases that data have been complemented/enhanced by data from zbMATH Open. This attempts to reflect the references listed in the original paper as accurately as possible without claiming completeness or a perfect matching.