Machine learning and design of experiments with an application to product innovation in the chemical industry. (English) Zbl 07563018
Summary: Industrial statistics plays a major role in the areas of both quality management and innovation. However, existing methodologies must be integrated with the latest tools from the field of Artificial Intelligence. To this end, a background on the joint application of Design of Experiments (DOE) and Machine Learning (ML) methodologies in industrial settings is presented here, along with a case study from the chemical industry. A DOE study is used to collect data, and two ML models are applied to predict responses which performance show an advantage over the traditional modeling approach. Emphasis is placed on causal investigation and quantification of prediction uncertainty, as these are crucial for an assessment of the goodness and robustness of the models developed. Within the scope of the case study, the models learned can be implemented in a semi-automatic system that can assist practitioners who are inexperienced in data analysis in the process of new product development.
MSC:
| 62-XX | Statistics |
References:
| [1] | Bagci, E.; Işık, B., Investigation of surface roughness in turning unidirectional gfrp composites by using rs methodology and ann, Int. J. Adv. Manuf. Technol., 31, 10-17 (2006) |
| [2] | Beck, M. W., Neuralnettools: visualization and analysis tools for neural networks, J. Stat. Softw., 85, 1 (2018) |
| [3] | Biranje, S. S.; Nathany, A.; Mehra, N.; Adivarekar, R., Optimisation of detergent ingredients for stain removal using statistical modelling, J. Surfactants. Deterg., 18, 949-956 (2015) |
| [4] | Bisgaard, S., The future of quality technology: from a manufacturing to a knowledge economy & from defects to innovations, Qual. Eng., 24, 30-36 (2012) |
| [5] | Box, G. E.P.; Behnken, D. W., Some new three level designs for the study of quantitative variables, Technometrics, 2, 455-475 (1960) · Zbl 0097.34406 |
| [6] | Box, G. E.P.; Liu, P. Y.T., Statistics as a catalyst to learning by scientific method part i-an example, J. Quality Technol., 31, 1-15 (1999) |
| [7] | Box, G. E.; Woodall, W. H., Innovation, quality engineering, and statistics, Qual. Eng., 24, 20-29 (2012) |
| [8] | Brecher, C.; Obdenbusch, M.; Buchsbaum, M., Optimized state estimation by application of machine learning, Prod. Eng., 11, 133-143 (2017) |
| [9] | Breiman, L., Random forests, Mach. Learn., 45, 5-32 (2001) · Zbl 1007.68152 |
| [10] | Cao, B.; Adutwum, L. A.; Oliynyk, A. O.; Luber, E. J.; Olsen, B. C.; Mar, A.; Buriak, J. M., How to optimize materials and devices via design of experiments and machine learning: demonstration using organic photovoltaics, ACS. Nano., 12, 7434-7444 (2018) |
| [11] | Çaydaş, U.; Hascalık, A., A study on surface roughness in abrasive waterjet machining process using artificial neural networks and regression analysis method, J. Mater. Process. Technol., 202, 574-582 (2008) |
| [12] | Chi, H.-M.; Ersoy, O. K.; Moskowitz, H.; Ward, J., Modeling and optimizing a vendor managed replenishment system using machine learning and genetic algorithms, Eur. J. Oper. Res., 180, 174-193 (2007) · Zbl 1114.90400 |
| [13] | Cox, D.; Kartsonaki, C.; Keogh, R. H., Big data: some statistical issues, Stat. Probab. Lett., 136, 111-115 (2018) · Zbl 1489.62409 |
| [14] | Domagalski, N. R.; Mack, B. C.; Tabora, J. E., Analysis of design of experiments with dynamic responses, Org. Process. Res. Dev., 19, 1667-1682 (2015) |
| [15] | Efron, B., Jackknife-after-bootstrap standard errors and influence functions, J. R. Stat. Soc.: Ser. B (Methodological), 54, 83-111 (1992) · Zbl 0782.62051 |
| [16] | Efron, B., Estimation and accuracy after model selection, J. Am. Stat. Assoc., 109, 991-1007 (2014) · Zbl 1368.62071 |
| [17] | Erzurumlu, T.; Oktem, H., Comparison of response surface model with neural network in determining the surface quality of moulded parts, Mater. Des., 28, 459-465 (2007) |
| [18] | Faraj, S.; Pachidi, S.; Sayegh, K., Working and organizing in the age of the learning algorithm, Information and Organization, 28, 62-70 (2018) |
| [19] | Fredj, N. B.; Amamou, R., Ground surface roughness prediction based upon experimental design and neural network models, Int. J. Adv. Manuf. Technol., 31, 24-36 (2006) |
| [20] | Freiesleben, J.; Keim, J.; Grutsch, M., Machine learning and design of experiments: alternative approaches or complementary methodologies for quality improvement?, Quality and Reliab. Eng. Int., 36, 1837-1848 (2020) |
| [21] | Friedman, J.; Hastie, T.; Tibshirani, R., The elements of statistical learning, 1 (2001), New York · Zbl 0973.62007 |
| [22] | Gal, Y. and Ghahramani, Z., Dropout as a bayesian approximation: Representing model uncertainty in deep learning, international conference on machine learning, 2016. pp. 1050-1059. |
| [23] | Garson, G. D., Interpreting neural-network connection weights, AI Expert, 6, 46-51 (1991) |
| [24] | Geurts, P.; Ernst, D.; Wehenkel, L., Extremely randomized trees, Mach. Learn., 63, 3-42 (2006) · Zbl 1470.68111 |
| [25] | Ghalandari, M.; Ziamolki, A.; Mosavi, A.; Shamshirband, S.; Chau, K.-W.; Bornassi, S., Aeromechanical optimization of first row compressor test stand blades using a hybrid machine learning model of genetic algorithm, artificial neural networks and design of experiments, Eng.g Appl. Comput. Fluid Mech., 13, 892-904 (2019) |
| [26] | Golkarnarenji, G.; Naebe, M.; Badii, K.; Milani, A. S.; Jazar, R. N.; Khayyam, H., A machine learning case study with limited data for prediction of carbon fiber mechanical properties, Computers Indust., 105, 123-132 (2019) |
| [27] | Hajjem, A.; Bellavance, F.; Larocque, D., Mixed-effects random forest for clustered data, J. Stat. Comput. Simul., 84, 1313-1328 (2014) · Zbl 1453.62543 |
| [28] | Hassanin, H.; Alkendi, Y.; Elsayed, M.; Essa, K.; Zweiri, Y., Controlling the properties of additively manufactured cellular structures using machine learning approaches, Adv. Eng. Mater., 22 (2020) |
| [29] | Hertlein, N.; Deshpande, S.; Venugopal, V.; Kumar, M.; Anand, S., Prediction of selective laser melting part quality using hybrid bayesian network, Additive Manufacturing, 32 (2020) |
| [30] | Hockman, K. K.; Jensen, W. A., Statisticians as innovation leaders, Qual. Eng., 28, 165-174 (2016) |
| [31] | Ibrić, S.; Djuriš, J.; Parojčić, J.; Djurić, Z., Artificial neural networks in evaluation and optimization of modified release solid dosage forms, Pharmaceutics, 4, 531-550 (2012) |
| [32] | Ivić, B.; Ibrić, S.; Cvetković, N.; Petrović, A.; Trajković, S.; Djurić, Z., Application of design of experiments and multilayer perceptrons neural network in the optimization of diclofenac sodium extended release tablets with carbopol \(#### 71\) g, Chemical and Pharmaceutical Bulletin, 58, 947-949 (2010) |
| [33] | Jaeckel, L.A., The infinitesimal jackknife. Bell Telephone Laboratories, 1972. |
| [34] | Jensen, W. A.; Montgomery, D. C.; Tsung, F.; Vining, G. G., 50 years of the journal of quality technology, J. Quality Technol., 50, 2-16 (2018) |
| [35] | Karazi, S.; Issa, A.; Brabazon, D., Comparison of ann and doe for the prediction of laser-machined micro-channel dimensions, Opt. Lasers. Eng., 47, 956-964 (2009) |
| [36] | Kim, D.-Y.; Kumar, V.; Kumar, U., Relationship between quality management practices and innovation, J. Oper. Manag., 30, 295-315 (2012) |
| [37] | Lakshminarayanan, B., Pritzel, A. and Blundell, C.. Simple and scalable predictive uncertainty estimation using deep ensembles, Advances in neural information processing systems, 2017. pp. 6402-6413. |
| [38] | Lasheras, F. S.; Vilán, J. V.; Nieto, P. G.; del Coz Díaz, J., The use of design of experiments to improve a neural network model in order to predict the thickness of the chromium layer in a hard chromium plating process, Math. Comput. Model., 52, 1169-1176 (2010) |
| [39] | Lou, H.; Chung, J. I.; Kiang, Y.-H.; Xiao, L.-Y.; Hageman, M. J., The application of machine learning algorithms in understanding the effect of core/shell technique on improving powder compactability, Int. J. Pharm., 555, 368-379 (2019) |
| [40] | Lucas, J. M., Which response surface design is best: A performance comparison of several types of quadratic response surface designs in symmetric regions, Technometrics, 18, 411-417 (1976) · Zbl 0342.62051 |
| [41] | Lujan-Moreno, G. A.; Howard, P. R.; Rojas, O. G.; Montgomery, D. C., Design of experiments and response surface methodology to tune machine learning hyperparameters, with a random forest case-study, Expert. Syst. Appl., 109, 195-205 (2018) |
| [42] | Luo, M. R., CIELAB, 1-7 (2015), Springer Berlin Heidelberg: Springer Berlin Heidelberg, Berlin, Heidelberg |
| [43] | Mee, R. W., Optimal three-level designs for response surfaces in spherical experimental regions, J. Quality Technol., 39, 340-354 (2007) |
| [44] | Montgomery, D. C., Experimental design for product and process design and development, J. R. Stat. Soc.: Ser. D (The Stat.), 48, 159-177 (1999) |
| [45] | Naftaly, U.; Intrator, N.; Horn, D., Optimal ensemble averaging of neural networks, Netw.: Comput. Neural Syst., 8, 283-296 (1997) · Zbl 0902.68169 |
| [46] | Olden, J. D.; Joy, M. K.; Death, R. G., An accurate comparison of methods for quantifying variable importance in artificial neural networks using simulated data, Ecol. Modell., 178, 389-397 (2004) |
| [47] | Özesmi, S. L.; Özesmi, U., An artificial neural network approach to spatial habitat modelling with interspecific interaction, Ecol. Modell., 116, 15-31 (1999) |
| [48] | Quaglio, M.; Roberts, L.; Jaapar, M. S.B.; Fraga, E. S.; Dua, V.; Galvanin, F., An artificial neural network approach to recognise kinetic models from experimental data, Comput. Chem. Eng., 135 (2020) |
| [49] | Radetzky, M.; Rosebrock, C.; Bracke, S., Approach to adapt manufacturing process parameters systematically based on machine learning algorithms, IFAC-PapersOnLine, 52, 1773-1778 (2019) |
| [50] | Rumelhart, D. E.; Hinton, G. E.; Williams, R. J., Learning representations by back-propagating errors, Nature, 323, 533-536 (1986) · Zbl 1369.68284 |
| [51] | Salmaso, L.; Pegoraro, L.; Giancristofaro, R. A.; Ceccato, R.; Bianchi, A.; Restello, S.; Scarabottolo, D., Design of experiments and machine learning to improve robustness of predictive maintenance with application to a real case study, Commun. Stat.-Simul. Comput., 1-13 (2019) |
| [52] | Santos, G.; Gomes, S.; Braga, V.; Braga, A.; Lima, V.; Teixeira, P.; Sá, J. C., Value creation through quality and innovation-a case study on portugal, The TQM J. (2019) |
| [53] | Schuetzenmeister, A. and Dufey, F., VCA: Variance Component Analysis, 2020. R package version 1.4.3. |
| [54] | Spedding, T. A.; Wang, Z., Study on modeling of wire edm process, J. Mater. Process. Technol., 69, 18-28 (1997) |
| [55] | Speiser, J. L.; Wolf, B. J.; Chung, D.; Karvellas, C. J.; Koch, D. G.; Durkalski, V. L., Bimm forest: A random forest method for modeling clustered and longitudinal binary outcomes, Chemometr. Intell. Lab. Syst., 185, 122-134 (2019) |
| [56] | Speiser, J. L.; Wolf, B. J.; Chung, D.; Karvellas, C. J.; Koch, D. G.; Durkalski, V. L., Bimm tree: a decision tree method for modeling clustered and longitudinal binary outcomes, Commun. Stat. - Simul. Comput., 49, 1004-1023 (2020) · Zbl 07552635 |
| [57] | Staelin, C.. Parameter selection for support vector machines, Hewlett-Packard Company, Tech. Rep. HPL-2002-354R1, 1, 2003. |
| [58] | Stroup, W. W., Generalized Linear Mixed Models: Modern Concepts, Methods and Applications (2012), CRC press, Boca Raton · Zbl 1281.62013 |
| [59] | Wager, S.; Hastie, T.; Efron, B., Confidence intervals for random forests: the jackknife and the infinitesimal jackknife, J. Machine Learning Res., 15, 1625-1651 (2014) · Zbl 1319.62132 |
| [60] | Wang, C.; Yang, Q.; Wang, J.; Zhao, J.; Wan, X.; Guo, Z.; Yang, Y., Application of support vector machine on controlling the silanol groups of silica xerogel with the aid of segmented continuous flow reactor, Chem. Eng. Sci., 199, 486-495 (2019) |
| [61] | Wong, P. K.; Gao, X. H.; Wong, K. I.; Vong, C. M., Efficient point-by-point engine calibration using machine learning and sequential design of experiment strategies, J. Franklin. Inst., 355, 1517-1538 (2018) · Zbl 1395.93435 |
| [62] | Wright, M. N.; Ziegler, A., ranger: A fast implementation of random forests for high dimensional data in C++ and R, J. Stat. Softw., 77, 1-17 (2017) |
| [63] | Yan, W.; Guo, Z.; Jia, X.; Kariwala, V.; Chen, T.; Yang, Y., Model-aided optimization and analysis of multi-component catalysts: application to selective hydrogenation of cinnamaldehyde, Chem. Eng. Sci., 76, 26-36 (2012) |
| [64] | Yusr, M. M., Innovation capability and its role in enhancing the relationship between tqm practices and innovation performance, Journal of Open Innovation: Technology, Market, and Complexity, 2, 6 (2016) |
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. In some cases that data have been complemented/enhanced by data from zbMATH Open. This attempts to reflect the references listed in the original paper as accurately as possible without claiming completeness or a perfect matching.
