×

MISO: mixed-integer surrogate optimization framework. (English) Zbl 1364.90230

Summary: We introduce MISO, the mixed-integer surrogate optimization framework. MISO aims at solving computationally expensive black-box optimization problems with mixed-integer variables. This type of optimization problem is encountered in many applications for which time consuming simulation codes must be run in order to obtain an objective function value. Examples include optimal reliability design and structural optimization. A single objective function evaluation may take from several minutes to hours or even days. Thus, only very few objective function evaluations are allowable during the optimization. The development of algorithms for this type of optimization problems has, however, rarely been addressed in the literature. Because the objective function is black-box, derivatives are not available and numerically approximating the derivatives requires a prohibitively large number of function evaluations. Therefore, we use computationally cheap surrogate models to approximate the expensive objective function and to decide at which points in the variable domain the expensive objective function should be evaluated. We develop a general surrogate model framework and show how sampling strategies of well-known surrogate model algorithms for continuous optimization can be modified for mixed-integer variables. We introduce two new algorithms that combine different sampling strategies and local search to obtain high-accuracy solutions. We compare MISO in numerical experiments to a genetic algorithm, NOMAD version 3.6.2, and SO-MI. The results show that MISO is in general more efficient than NOMAD and the genetic algorithm with respect to finding improved solutions within a limited budget of allowable evaluations. The performance of MISO depends on the chosen sampling strategy. The MISO algorithm that combines a coordinate perturbation search with a target value strategy and a local search performs best among all algorithms.

MSC:

90C11 Mixed integer programming
90C26 Nonconvex programming, global optimization
90C56 Derivative-free methods and methods using generalized derivatives
Full Text: DOI

References:

[1] Abramson M, Audet C, Chrissis J, Walston J (2009) Mesh adaptive direct search algorithms for mixed variable optimization. Optim Lett 3:35-47 · Zbl 1154.90551 · doi:10.1007/s11590-008-0089-2
[2] Booker A, Dennis J Jr, Frank P, Serafini D, Torczon V, Trosset M (1999) A rigorous framework for optimization of expensive functions by surrogates. Struct Multidiscip Optim 17:1-13 · doi:10.1007/BF01197708
[3] Conn A, Scheinberg K, Vicente L (2009) Introduction to Derivative-Free Optimization. SIAM · Zbl 1163.49001
[4] Currie J, Wilson D (2012) Foundations of computer-aided process operations. OPTI: lowering the barrier between open source optimizers and the industrial MATLAB user. Savannah, Georgia, USA · Zbl 1241.90192
[5] Davis E, Ierapetritou M (2009) Kriging based method for the solution of mixed-integer nonlinear programs containing black-box functions. J Glob Optim 43:191-205 · Zbl 1179.90238 · doi:10.1007/s10898-007-9217-2
[6] Forrester A, Sóbester A, Keane A (2008) Engineering design via surrogate modelling—a practical guide. Wiley, Chichester · doi:10.1002/9780470770801
[7] Friedman J (1991) Multivariate adaptive regression splines. Ann Stat 19:1-141 · Zbl 0765.62064 · doi:10.1214/aos/1176347963
[8] Giunta A, Balabanov V, Haim D, Grossman B, Mason W, Watson L, Haftka R (1997) Aircraft multidisciplinary design optimisation using design of experiments theory and response surface modelling. Aeronaut J 101:347-356
[9] Glaz B, Friedmann P, Liu L (2008) Surrogate based optimization of helicopter rotor blades for vibration reduction in forward flight. Struct Multidiscip Optim 35:341-363 · doi:10.1007/s00158-007-0137-z
[10] Goel T, Haftka RT, Shyy W, Queipo NV (2007) Ensemble of surrogates. Struct Multidiscip Optim 33:199-216 · doi:10.1007/s00158-006-0051-9
[11] Gutmann H (2001) A radial basis function method for global optimization. J Glob Optim 19:201-227 · Zbl 0972.90055 · doi:10.1023/A:1011255519438
[12] Hemker T, Fowler K, Farthing M, von Stryk O (2008) A mixed-integer simulation-based optimization approach with surrogate functions in water resources management. Optim Eng 9:341-360 · Zbl 1419.90007 · doi:10.1007/s11081-008-9048-0
[13] Holmström K (2008a) An adaptive radial basis algorithm (ARBF) for expensive black-box global optimization. J Glob Optim 41:447-464 · Zbl 1152.90609 · doi:10.1007/s10898-007-9256-8
[14] Holmström K (2008b) An adaptive radial basis algorithm (ARBF) for expensive black-box mixed-integer global optimization. J Glob Optim 9:311-339 · Zbl 1400.90226
[15] Jones D, Schonlau M, Welch W (1998) Efficient global optimization of expensive black-box functions. J Glob Optim 13:455-492 · Zbl 0917.90270 · doi:10.1023/A:1008306431147
[16] Koziel S, Leifsson L (2013) Surroagte-based modeling and optimization: applications in engineering. Springer, New York · Zbl 1269.00012 · doi:10.1007/978-1-4614-7551-4
[17] Le Digabel S (2011) Algorithm 909: NOMAD: nonlinear optimization with the mads algorithm. ACM Trans Math Softw 37:44 · Zbl 1365.65172 · doi:10.1145/1916461.1916468
[18] Li R, Emmerich M, Eggermont J, Bovenkamp E, Back T, Dijkstra J, Reiber H (2008) Metamodel-assisted mixed-integer evolution strategies and their applications to intravascular ultrasound image analysis. In: IEEE World Congress on Computational Intelligence, IEEE, pp 2764-2771 · Zbl 1365.65172
[19] Marsden A, Wang M, Dennis J Jr, Moin P (2004) Optimal aeroacoustic shape design using the surrogate management framework. Optim Eng 5:235-262 · Zbl 1116.90417 · doi:10.1023/B:OPTE.0000033376.89159.65
[20] Moré J, Wild S (2009) Benchmarking derivative-free optimization algorithms. SIAM J Optim 20:172-191 · Zbl 1187.90319 · doi:10.1137/080724083
[21] Müller J (2014) MATSuMoTo: The MATLAB surrogate model toolbox for computationally expensive black-box global optimization problems. arXiv:14044261 · Zbl 1187.90319
[22] Müller J, Shoemaker C, Piché R (2013a) SO-I: a surrogate model algorithm for expensive nonlinear integer programming problems including global optimization applications. J Glob Optim 59:865-889. doi:10.1007/s10,898-013-0101-y · Zbl 1305.90305 · doi:10.1007/s10,898-013-0101-y
[23] Müller J, Shoemaker C, Piché R (2013b) SO-MI: a surrogate model algorithm for computationally expensive nonlinear mixed-integer black-box global optimization problems. Comput Oper Res 40:1383-1400 · Zbl 1352.90067 · doi:10.1016/j.cor.2012.08.022
[24] Müller J, Piché R (2011) Mixture surrogate models based on Dempster-Shafer theory for global optimization problems. J Glob Optim 51:79-104 · Zbl 1230.90155 · doi:10.1007/s10898-010-9620-y
[25] Müller J, Shoemaker C (2014) Influence of ensemble surrogate models and sampling strategy on the solution quality of algorithms for computationally expensive black-box global optimization problems. J Glob Optim 60:123-144. doi:10.1007/s10898-014-0184-0 · Zbl 1312.90064 · doi:10.1007/s10898-014-0184-0
[26] Myers R, Montgomery D (1995) Response surface methodology: process and product optimization using designed experiments. Wiley-Interscience Publication, Hoboken · Zbl 1161.62392
[27] Powell, M.; Light, WA (ed.), Advances in numerical analysis, vol. 2: wavelets, subdivision algorithms and radial basis functions, 105-210 (1992), Oxford · Zbl 0787.65005
[28] Rashid S, Ambani S, Cetinkaya E (2012) An adaptive multiquadric radial basis function method for expensive black-box mixed-integer nonlinear constrained optimization. Eng Optim 45:185-206. doi:10.1080/0305215X.2012.665450 · doi:10.1080/0305215X.2012.665450
[29] Regis R, Shoemaker C (2007) A stochastic radial basis function method for the global optimization of expensive functions. INFORMS J Comput 19:497-509 · Zbl 1241.90192 · doi:10.1287/ijoc.1060.0182
[30] Regis R, Shoemaker C (2009) Parallel stochastic global optimization using radial basis functions. INFORMS J Comput 21:411-426 · Zbl 1243.90160 · doi:10.1287/ijoc.1090.0325
[31] Regis R, Shoemaker C (2013) Combining radial basis function surrogates and dynamic coordinate search in high-dimensional expensive black-box optimization. Eng Optim 45:529-555 · doi:10.1080/0305215X.2012.687731
[32] Simpson T, Mauery T, Korte J, Mistree F (2001) Kriging metamodels for global approximation in simulation-based multidisciplinary design optimization. AIAA J 39:2233-2241 · doi:10.2514/2.1234
[33] Viana F, Haftka R, Steffen V (2009) Multiple surrogates: how cross-validation errors can help us to obtain the best predictor. Struct Multidiscip Optim 39:439-457 · doi:10.1007/s00158-008-0338-0
[34] Wild S, Regis R, Shoemaker C (2007) ORBIT: optimization by radial basis function interpolation in trust-regions. SIAM J Sci Comput 30:3197-3219 · Zbl 1178.65065 · doi:10.1137/070691814
[35] Wild S, Shoemaker C (2013) Global convergence of radial basis function trust-region algorithms for derivative-free optimization. SIAM Rev 55:349-371 · Zbl 1270.65028 · doi:10.1137/120902434
[36] Zhuang, L.; Tang, K.; Jin, Y.; Hea, Yin (ed.), Metamodel assisted mixed-integer evolution strategies based on Kendall rank correlation coefficient, 366-375 (2013), Berlin
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. In some cases that data have been complemented/enhanced by data from zbMATH Open. This attempts to reflect the references listed in the original paper as accurately as possible without claiming completeness or a perfect matching.