Global optimization of general constrained grey-box models: new method and its application to constrained PDEs for pressure swing adsorption. (English) Zbl 1359.90101
Summary: This paper introduces a novel methodology for the global optimization of general constrained grey-box problems. A grey-box problem may contain a combination of black-box constraints and constraints with a known functional form. The novel features of this work include (i) the selection of initial samples through a subset selection optimization problem from a large number of faster low-fidelity model samples (when a low-fidelity model is available), (ii) the exploration of a diverse set of interpolating and non-interpolating functional forms for representing the objective function and each of the constraints, (iii) the global optimization of the parameter estimation of surrogate functions and the global optimization of the constrained grey-box formulation, and (iv) the updating of variable bounds based on a clustering technique. The performance of the algorithm is presented for a set of case studies representing an expensive non-linear algebraic partial differential equation simulation of a pressure swing adsorption system for \(\mathrm{CO}_{2}\). We address three significant sources of variability and their effects on the consistency and reliability of the algorithm: (i) the initial sampling variability, (ii) the type of surrogate function, and (iii) global versus local optimization of the surrogate function parameter estimation and overall surrogate constrained grey-box problem. It is shown that globally optimizing the parameters in the parameter estimation model, and globally optimizing the constrained grey-box formulation has a significant impact on the performance. The effect of sampling variability is mitigated by a two-stage sampling approach which exploits information from reduced-order models. Finally, the proposed global optimization approach is compared to existing constrained derivative-free optimization algorithms.
MSC:
| 90C26 | Nonconvex programming, global optimization |
| 90C56 | Derivative-free methods and methods using generalized derivatives |
Keywords:
derivative-free optimization; kriging; quadratic; constrained optimization; sampling reduction; global optimizationSoftware:
NOMAD; ANTIGONE; MultiMin; BOBYQA; MLMSRBF; AQUARS; CONOPT; EGO; TOMLAB; GloMIQO; ORBIT; NLopt; OrthoMADSReferences:
| [1] | Audet, C., Bechard, V., Chaouki, J.: Spent potliner treatment process optimization using a MADS algorithm. Optim. Eng. 9(2), 143-160 (2008) · Zbl 1167.92039 · doi:10.1007/s11081-007-9030-2 |
| [2] | Bartholomew-Biggs, M.C., Parkhurst, S.C., Wilson, S.P.: Using DIRECT to solve an aircraft routing problem. Comput. Optim. Appl. 21(3), 311-323 (2002) · Zbl 1017.90133 · doi:10.1023/A:1013729320435 |
| [3] | Boukouvala, F., Ierapetritou, M.G.: Surrogate-based optimization of expensive flowsheet modeling for continuous pharmaceutical manufacturing. J. Pharm. Innov. 8(2), 131-145 (2013) · doi:10.1007/s12247-013-9154-1 |
| [4] | Caballero, J.A., Grossmann, I.E.: An algorithm for the use of surrogate models in modular flowsheet optimization. AIChE J. 54(10), 2633-2650 (2008) · doi:10.1002/aic.11579 |
| [5] | Egea, J.A., Rodriguez-Fernandez, M., Banga, J.R., Marti, R.: Scatter search for chemical and bio-process optimization. J. Glob. Optim. 37(3), 481-503 (2007) · Zbl 1108.92001 · doi:10.1007/s10898-006-9075-3 |
| [6] | Fahmi, I., Cremaschi, S.: Process synthesis of biodiesel production plant using artificial neural networks as the surrogate models. Comput. Chem. Eng. 46, 105-123 (2012) · doi:10.1016/j.compchemeng.2012.06.006 |
| [7] | Fowler, K.R., Reese, J.P., Kees, C.E., Dennis Jr, J.E., Kelley, C.T., Miller, C.T., Audet, C., Booker, A.J., Couture, G., Darwin, R.W., Farthing, M.W., Finkel, D.E., Gablonsky, J.M., Gray, G., Kolda, T.G.: Comparison of derivative-free optimization methods for groundwater supply and hydraulic capture community problems. Adv. Water Resour. 31(5), 743-757 (2008) · doi:10.1016/j.advwatres.2008.01.010 |
| [8] | Graciano, J.E.A., Roux, G.A.C.L.: Improvements in surrogate models for process synthesis. Application to water network system design. Comput. Chem. Eng. 59, 197-210 (2013) · doi:10.1016/j.compchemeng.2013.05.024 |
| [9] | Hemker, T., Fowler, K., Farthing, M., Stryk, O.: A mixed-integer simulation-based optimization approach with surrogate functions in water resources management. Optim. Eng. 9(4), 341-360 (2008) · Zbl 1419.90007 · doi:10.1007/s11081-008-9048-0 |
| [10] | Henao, C.A., Maravelias, C.T.: Surrogate-based superstructure optimization framework. AIChE J. 57(5), 1216-1232 (2011) · doi:10.1002/aic.12341 |
| [11] | Kleijnen, J.P.C., van Beers, W., van Nieuwenhuyse, I.: Constrained optimization in expensive simulation: Novel approach. Eur. J. Oper. Res. 202(1), 164-174 (2010) · Zbl 1189.90156 · doi:10.1016/j.ejor.2009.05.002 |
| [12] | Wan, X.T., Pekny, J.F., Reklaitis, G.V.: Simulation-based optimization with surrogate models—application to supply chain management. Comput. Chem. Eng. 29(6), 1317-1328 (2005) · doi:10.1016/j.compchemeng.2005.02.018 |
| [13] | Espinet, A., Shoemaker, C., Doughty, C.: Estimation of plume distribution for carbon sequestration using parameter estimation with limited monitoring data. Water Resour. Res. 49(7), 4442-4464 (2013) · doi:10.1002/wrcr.20326 |
| [14] | Hasan, M.M.F., Baliban, R.C., Elia, J.A., Floudas, C.A.: Modeling, simulation, and pptimization of postcombustion \[{\rm CO}_2\] CO2 capture for variable feed concentration and flow rate. 2. Pressure swing adsorption and vacuum swing adsorption processes. Ind. Eng. Chem. Res. 51(48), 15665-15682 (2013) · doi:10.1021/ie301572n |
| [15] | Hasan, M.M.F., Baliban, R.C., Elia, J.A., Floudas, C.A.: Modeling, simulation, and optimization of postcombustion \[\text{ CO }_2\] CO2 capture for variable feed concentration and flow rate. 1. Chemical absorption and membrane processes. Ind. Eng. Chem. Res. 51(48), 15642-15664 (2013) · doi:10.1021/ie301571d |
| [16] | Hasan, M.M.F., Boukouvala, F., First, E.L., Floudas, C.A.: Nationwide, regional, and statewide \[\text{ CO }_2\] CO2 capture, utilization, and sequestration supply chain network optimization. Ind. Eng. Chem. Res. 53(18), 7489-7506 (2014) · doi:10.1021/ie402931c |
| [17] | Li, S., Feng, L., Benner, P., Seidel-Morgenstern, A.: Using surrogate models for efficient optimization of simulated moving bed chromatography. Comput. Chem. Eng. 67, 121-132 (2014) · doi:10.1016/j.compchemeng.2014.03.024 |
| [18] | Forrester, A.I.J., Sóbester, A., Keane, A.J.: Engineering Design Via Surrogate Modelling—A Practical Guide. Wiley, Chichester (2008) · doi:10.1002/9780470770801 |
| [19] | Conn, A.R., Scheinberg, K., Vicente, L.N.: Introduction to Derivative-Free Optimization. MPS-SIAM Series on Optimization, vol. 8. SIAM, Philadelphia (2009) · Zbl 1163.49001 · doi:10.1137/1.9780898718768 |
| [20] | Martelli, E., Amaldi, E.: PGS-COM: a hybrid method for constrained non-smooth black-box optimization problems: brief review, novel algorithm and comparative evaluation. Comput. Chem. Eng. 63, 108-139 (2014) · doi:10.1016/j.compchemeng.2013.12.014 |
| [21] | Rios, L.M., Sahinidis, N.V.: Derivative-free optimization: a review of algorithms and comparison of software implementations. J. Glob. Optim. 56(3), 1247-1293 (2013) · Zbl 1272.90116 · doi:10.1007/s10898-012-9951-y |
| [22] | Kolda, T.G., Lewis, R.M., Torczon, V.: Optimization by direct search: new perspectives on some classical and modern methods. SIAM Rev. 45(3), 385-482 (2003) · Zbl 1059.90146 · doi:10.1137/S003614450242889 |
| [23] | Bjorkman, M., Holmstrom, K.: Global optimization of costly nonconvex functions using radial basis functions. Optim. Eng. 1(4), 373-397 (2000) · Zbl 1035.90061 · doi:10.1023/A:1011584207202 |
| [24] | Booker, A.J., Dennis, J.E., Frank, P.D., Serafini, D.B., Torczon, V., Trosset, M.W.: A rigorous framework for optimization of expensive functions by surrogates. Struct. Multidiscip. Optim. 17(1), 1-13 (1999) · doi:10.1007/BF01197708 |
| [25] | Boukouvala, F., Ierapetritou, M.G.: Derivative-free optimization for expensive constrained problems using a novel expected improvement objective function. AIChE J. 60(7), 2462-2474 (2014) · doi:10.1002/aic.14442 |
| [26] | Conn, A.R., Le Digabel, S.: Use of quadratic models with mesh-adaptive direct search for constrained black box optimization. Optim. Methods Softw. 28(1), 139-158 (2013) · Zbl 1270.90073 · doi:10.1080/10556788.2011.623162 |
| [27] | Forrester, A.I.J., Keane, A.J.: Recent advances in surrogate-based optimization. Prog. Aerosp. Sci. 45(1), 50-79 (2009) · doi:10.1016/j.paerosci.2008.11.001 |
| [28] | Jakobsson, S., Patriksson, M., Rudholm, J., Wojciechowski, A.: A method for simulation based optimization using radial basis functions. Optim. Engi. 11(4), 501-532 (2010) · Zbl 1243.65068 · doi:10.1007/s11081-009-9087-1 |
| [29] | Jones, D.R.: A taxonomy of global optimization methods based on response surfaces. J. Global Optim. 21(4), 345-383 (2001) · Zbl 1172.90492 · doi:10.1023/A:1012771025575 |
| [30] | Jones, D.R., Schonlau, M., Welch, W.J.: Efficient global optimization of expensive black-box functions. J. Global Optim. 13(4), 455-492 (1998) · Zbl 0917.90270 · doi:10.1023/A:1008306431147 |
| [31] | Regis, R.G.: Constrained optimization by radial basis function interpolation for high-dimensional expensive black-box problems with infeasible initial points. Eng. Optim. 46(2), 218-243 (2014) · doi:10.1080/0305215X.2013.765000 |
| [32] | Regis, R.G., Shoemaker, C.A.: Constrained global optimization of expensive black box functions using radial basis functions. J. Glob. Optim. 31(1), 153-171 (2005) · Zbl 1274.90511 · doi:10.1007/s10898-004-0570-0 |
| [33] | Yao, W., Chen, X.Q., Huang, Y.Y., van Tooren, M.: A surrogate-based optimization method with RBF neural network enhanced by linear interpolation and hybrid infill strategy. Optim. Methods Softw. 29(2), 406-429 (2014) · Zbl 1285.90072 · doi:10.1080/10556788.2013.777722 |
| [34] | Muller, J., Shoemaker, C.A.: Influence ensemble surrogate models and sampling strategy on the solution quality of algorithms for computationally expensive black-box global optimization methods. J. Glob. Optim. 60(2), 123-144 (2014) · Zbl 1312.90064 · doi:10.1007/s10898-014-0184-0 |
| [35] | Viana, F.A.C., Haftka, R.T., Watson, L.T.: Efficient global optimization algorithm assisted by multiple surrogate techniques. J. Glob. Optim. 56(2), 669-689 (2013) · Zbl 1275.90072 · doi:10.1007/s10898-012-9892-5 |
| [36] | Davis, E., Ierapetritou, M.: A kriging method for the solution of nonlinear programs with black-box functions. AIChE J. 53(8), 2001-2012 (2007) · doi:10.1002/aic.11228 |
| [37] | Floudas, C.A.: Deterministic Global Optimization, vol. 37. Springer, Berlin (1999) |
| [38] | Floudas, C.A., Gounaris, C.E.: A review of recent advances in global optimization. J. Glob. Optim. 45(1), 3-38 (2009) · Zbl 1180.90245 · doi:10.1007/s10898-008-9332-8 |
| [39] | First, E.L., Hasan, M.M.F., Floudas, C.A.: Discovery of novel zeolites for natural gas purification through combined material screening and process optimization. AICHE J. 60(5), 1767-1785 (2014) · doi:10.1002/aic.14441 |
| [40] | Hasan, M.M.F., First, E.L., Floudas, C.A.: Cost-effective \[\text{ CO }_2\] CO2 capture based on in silico screening of zeolites and process optimization. Phys. Chem. Chem. Phys. 15(40), 17601-17618 (2013) · doi:10.1039/c3cp53627k |
| [41] | Abramson, M.: Pattern Search Algorithms for Mixed Variable General Constrained Optimization Problems. Rice University, Houston (2002) |
| [42] | Audet, C., Dennis, J.: Mesh adaptive direct search algorithms for constrained optimization. SIAM J. Optim. 17(1), 188-217 (2006) · Zbl 1112.90078 · doi:10.1137/040603371 |
| [43] | Audet, C., Dennis Jr, J.E.: A progressive barrier for derivative-free nonlinear programming. SIAM J. Optim. 20(1), 445-472 (2009) · Zbl 1187.90266 · doi:10.1137/070692662 |
| [44] | Holmstrom, K., Quttineh, N.-H., Edvall, M.M.: An adaptive radial basis algorithm (ARBF) for expensive black-box mixed-integer constrained global optimization. Optim. Eng. 9(4), 311-339 (2008) · Zbl 1400.90226 · doi:10.1007/s11081-008-9037-3 |
| [45] | Audet, C., Dennis Jr, J.E.: A pattern search filter method for nonlinear programming without derivatives. SIAM J. Optim. 14(4), 980-1010 (2004) · Zbl 1073.90066 · doi:10.1137/S105262340138983X |
| [46] | Parr, J.M., Keane, A.J., Forrester, A.I.J., Holden, C.M.E.: Infill sampling criteria for surrogate-based optimization with constraint handling. Eng. Optim. 44(10), 1147-1166 (2012) · Zbl 1250.90089 · doi:10.1080/0305215X.2011.637556 |
| [47] | Sasena, M.J., Papalambros, P., Goovaerts, P.: Exploration of metamodeling sampling criteria for constrained global optimization. Eng. Optim. 34(3), 263-278 (2002) · doi:10.1080/03052150211751 |
| [48] | Regis, R.G.: Stochastic radial basis function algorithms for large-scale optimization involving expensive black-box objective and constraint functions. Comput. Oper. Res. 38(5), 837-853 (2011) · Zbl 1434.90109 · doi:10.1016/j.cor.2010.09.013 |
| [49] | Abramson, M., Audet, C., Dennis, J., Digabel, S.: OrthoMADS: a deterministic MADS instance with orthogonal directions. SIAM J. Optim. 20(2), 948-966 (2009) · Zbl 1189.90202 · doi:10.1137/080716980 |
| [50] | Audet, C., Bechard, V., Digabel, S.: Nonsmooth optimization through mesh adaptive direct search and variable neighborhood search. J. Glob. Optim. 41(2), 299-318 (2008) · Zbl 1157.90535 · doi:10.1007/s10898-007-9234-1 |
| [51] | Kolda, T.G., Lewis, R.M., Torczon, V.: Optimization by direct search: new perspectives on some classical and modern methods. SIAM Rev. 45(3), 385-482 (2003) · Zbl 1059.90146 · doi:10.1137/S003614450242889 |
| [52] | Wild, S.M., Regis, R.G., Shoemaker, C.A.: Orbit: optimization by radial basis function interpolation in trust-regions. SIAM J. Sci. Comput. 30(6), 3197-3219 (2008) · Zbl 1178.65065 · doi:10.1137/070691814 |
| [53] | Sankaran, S., Audet, C., Marsden, A.L.: A method for stochastic constrained optimization using derivative-free surrogate pattern search and collocation. J. Comput. Phys. 229(12), 4664-4682 (2010) · Zbl 1193.65100 · doi:10.1016/j.jcp.2010.03.005 |
| [54] | Le Digabel, S.: Algorithm 909: NOMAD: nonlinear optimization with the MADS algorithm. ACM Trans. Math. Softw. (TOMS) 37(4), 1-15 (2011) · Zbl 1365.65172 · doi:10.1145/1916461.1916468 |
| [55] | Oeuvray, R.: Trust-region methods based on radial basis functions with application to biomedical imaging. PhD in Mathematics, Ecole Polytechnique Federale de Lausanne (2005) · Zbl 1187.90266 |
| [56] | Powell, M.J.D.: The BOBYQA Algorithm for Bound Constrained Optimization Without Derivatives. Department of Applied Mathematics and Theoretical Physics, University of Cambridge, Cambridge (2009) |
| [57] | Regis, R.G., Shoemaker, C.A.: Parallel radial basis function methods for the global optimization of expensive functions. Eur. J. Oper. Res. 182(2), 514-535 (2007) · Zbl 1178.90279 · doi:10.1016/j.ejor.2006.08.040 |
| [58] | Regis, R.G., Shoemaker, C.A.: A stochastic radial basis function method for the global optimization of expensive functions. INFORMS J. Comput. 19(4), 497-509 (2007) · Zbl 1241.90192 · doi:10.1287/ijoc.1060.0182 |
| [59] | Regis, R.G., Shoemaker, C.A.: Improved strategies for radial basis function methods for global optimization. J. Glob. Optim. 37(1), 113-135 (2007) · Zbl 1149.90120 · doi:10.1007/s10898-006-9040-1 |
| [60] | Regis, R.G., Shoemaker, C.A.: A quasi-multistart framework for global optimization of expensive functions using response surface models. J. Glob. Optim. 56(4), 1719-1753 (2013) · Zbl 1275.90068 · doi:10.1007/s10898-012-9940-1 |
| [61] | Powell, MJD; Gomez, S. (ed.); Hennart, J-P (ed.), A direct search optimization method that models the objective and constraint functions by linear interpolation, 51-67 (1994), Berlin · Zbl 0826.90108 · doi:10.1007/978-94-015-8330-5_4 |
| [62] | Quan, N., Yin, J., Ng, S.H., Lee, L.H.: Simulation optimization via kriging: a sequential search using expected improvement with computing budget constraints. IIE Trans. 45(7), 763-780 (2013) · doi:10.1080/0740817X.2012.706377 |
| [63] | Torn, A., Zilinskas, A.: Global optimization. In: Lecture Notes in Computer Science, vol. 350. Springer, Berlin (1989) · Zbl 0752.90075 |
| [64] | Regis, R.G.: Evolutionary programming for high-dimensional constrained expensive black-box optimization using radial basis functions. IEEE Trans. Evol. Comput. 18(3), 326-347 (2014). doi:10.1109/TEVC.2013.2262111 · doi:10.1109/TEVC.2013.2262111 |
| [65] | Misener, R., Floudas, C.A.: Global optimization of mixed-integer models with quadratic and signomial functions: a review. Appl. Comput. Math. 11, 317-336 (2012) · Zbl 1292.90239 |
| [66] | Misener, R., Floudas, C.A.: GloMIQO: global mixed-integer quadratic optimizer. J. Glob. Optim. 57(1), 3-50 (2013) · Zbl 1272.90034 · doi:10.1007/s10898-012-9874-7 |
| [67] | Misener, R., Floudas, C.: ANTIGONE: algorithms for continuous/integer global optimization of nonlinear equations. J. Glob. Optim. 59(2-3), 503-526 (2014) · Zbl 1301.90063 · doi:10.1007/s10898-014-0166-2 |
| [68] | Mckay, M.D., Beckman, R.J., Conover, W.J.: A comparison of three methods for selecting values of input variables in the analysis of output from a computer code. Technometrics 42(1), 55-61 (2000) · Zbl 0415.62011 · doi:10.1080/00401706.2000.10485979 |
| [69] | Kahrs, O., Marquardt, W.: The validity domain of hybrid models and its application in process optimization. Chem. Eng. Process. 46(11), 1054-1066 (2007) · doi:10.1016/j.cep.2007.02.031 |
| [70] | Sobieszczanski-Sobieski, J., Haftka, R.T.: Multidisciplinary aerospace design optimization: survey of recent developments. Struct. Optim. 14(1), 1-23 (1997). doi:10.1007/BF01197554 · doi:10.1007/BF01197554 |
| [71] | Willcox, K., Peraire, J.: Balanced model reduction via the proper orthogonal decomposition. AIAA J. 40(11), 2323-2330 (2002) · doi:10.2514/2.1570 |
| [72] | Lucia, D.J., Beran, P.S., Silva, W.A.: Reduced-order modeling: new approaches for computational physics. Prog. Aerosp. Sci. 40(1-2), 51-117 (2004) · doi:10.1016/j.paerosci.2003.12.001 |
| [73] | Li, Z., Floudas, C.A.: Optimal scenario reduction framework based on distance of uncertainty distribution and output performance: I. Single reduction via mixed integer linear optimization. Comput. Chem. Eng. 70, 50-65 (2014) · doi:10.1016/j.compchemeng.2014.03.019 |
| [74] | Cressie, N.: Statistics for Spatial Data. Wiley Series in Probability and Statistics. Wiley-Interscience, New York (1993) · Zbl 0799.62002 |
| [75] | Kleijnen, J.P.C.: Kriging metamodeling in simulation: a review. Eur. J. Oper. Res. 192(3), 707-716 (2009) · Zbl 1157.90544 · doi:10.1016/j.ejor.2007.10.013 |
| [76] | Sacks, J., Welch, W.J., Toby, J.M., Wynn, H.P.: Design and analysis of computer experiments. Stat. Sci. 4(4), 409-423 (1989) · Zbl 0955.62619 · doi:10.1214/ss/1177012413 |
| [77] | Myers, R.H., Montgomery, D.C.: Response Surface Methodology: Process and Product in Optimization Using Designed Experiments. Wiley, New York (1995) · Zbl 1161.62392 |
| [78] | Bjork, K.-M., Lindberg, P.O., Westerlund, T.: Some convexifications in global optimization of problems containing signomial terms. Comput. Chem. Eng. 27(5), 669-679 (2003) · doi:10.1016/S0098-1354(02)00254-5 |
| [79] | Gramacy, R.B., Lee, H.K.H.: Optimization Under Unknown Constraints. University of Cambridge, Cambridge (2010) |
| [80] | Lin, L.-C., Berger, A., Martin, R., Kim, J., Swisher, J., Jariwala, K., Rycroft, C., Bhown, A., Deem, M., Haranczyk, M., Smit, B.: In silico screening of carbon-capture materials. Nat. Mater. 11(7), 633-641 (2012) · doi:10.1038/nmat3336 |
| [81] | Siriwardane, R.V., Shen, M.-S., Fisher, E.P., Poston, J.A.: Adsorption of \[\text{ CO }_2\] CO2 on molecular sieves and activated carbon. Energy Fuels 15(2), 279-284 (2001) · doi:10.1021/ef000241s |
| [82] | Zhang, J., Webley, P.A., Xiao, P.: Effect of process parameters on power requirements of vacuum swing adsorption technology for \[\text{ CO }_2\] CO2 capture from flue gas. Energy Convers. Manag. 49(2), 346-356 (2008) · doi:10.1016/j.enconman.2007.06.007 |
| [83] | Drud, A.: CONOPT—a large-scale GRG code. ORSA J. Comput. 6, 207-216 (1992) · Zbl 0806.90113 · doi:10.1287/ijoc.6.2.207 |
| [84] | Egea, J.A., Martí, R., Banga, J.R.: An evolutionary method for complex-process optimization. Comput. Oper. Res. 37(2), 315-324 (2010) · Zbl 1175.90427 · doi:10.1016/j.cor.2009.05.003 |
| [85] | Johnson, S.G.: The NLopt nonlinear-optimization package. http://ab-initio.mit.edu/nlopt |
| [86] | Holmstrom, K., Goran, A.O., Edvall, M.M.: Users Guide for TOMLAB CGO. http://tomopt.com/docs/TOMLAB_CGO.pdf (2008) |
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. In some cases that data have been complemented/enhanced by data from zbMATH Open. This attempts to reflect the references listed in the original paper as accurately as possible without claiming completeness or a perfect matching.
